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ABSTRACT 

This handbook was designed to help Australian 
technical, adult, and further education (TAFE) teachers, especially 
those in vocational education areas, to assess and evaluate students. 
The guide contains seven chapters, each written so it can stand 
alone. At the beginning of each chapter is an explanation of what is 
to follow and a statement of the objectives that the chapter aims to 
cover. Chapters conclude with self ^-assessment exercises for readers 
and, where appropriate, suggested answers to the exercises are 
provided. The guide covers the following subjects: introduction to 
assessment, objectives of assessment, types of questions, test 
specifications, item analysis, validity and reliability of assessment 
instruments, and reporting results. Numerous examples are given 
throughout the chapters. (KC) 
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PREFACE 



A number of good books about the techniques of student assessment 
are available on the Australian market. However, with only a few 
exceptions, these are directed at primary and secondary school 
teachers and their students* 

This book is aimed at a different audience; it is directed at 
technical and further education teachers, particularly at those 
teachers working in the vocational areas* It also aims to meet 
the needs of trainers working in commerce and industry who are 
increasingly being called up to improve the validity of their 
assessment procedures* 

Readers with a knowledge of other texts on the subject will note 
that although the ground covered is familiar, new ideas and new 
emphases Ate presented* Practical assessment receives special 
attention as do new approaches to describing student performance* 

Each Chapter has been written so that it can stand by itself* 
Chapters can therefore be read and studied independently of each 
other * 

At the beginning of a Chapter there is an explanation of what is 
to follow ana a statement of the objectives that the Chapter aims 
to cover* Chapters conclude with self ^-assessment exercises for 
the readers and, where appropriate, the suggested answers to the 
exercises are also provided* 



Peter Thomson 

Adelaide, 1986 
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CHllPTKR I 
IMTRODUCyiON TO ASSBSSMBNT 



NHAT THIS CHAPTER IS ABOUT 

This chapter has been written to present a general understanding 
of what is meant by assessment * 

Assessments are very much a part of our daily lives. We are 
constantly asking ourselves questions involving assessment like 
•will I get to work on time ?' 'Is that something I can afford?' 
•Is my tennis game improving?' 

The chapter addresses the problem of definition. It provides an 
educational basis for the term 'assessment' and explains how it 
differs from 'evaluation'. 

It should be remembered that this is an introductory chapter and 
that many of the ideas or concepts mentioned will be taken up in 
greater detail later. At this stage it is important to become 
familiar with these concepts and idease 



OBJECTIVES 

After completing this chapter you should be able to: 

explain the meaning of the term 'aeeeeement ' 

explain the meaning of the term 'evaluation' 

• distinguish between 'measurement' and 
' {^valuation ' 

Icnow that in education^ measurements are usually 
done using tests which may be of a variety of 
types 

distinguish between 'formative' and 'summative' 
evaluation 

distinguish between evaluation studies and 
research studies 

state the difference between 'norm^ref erenced' and 
'criterion^ref erenced' tests and list the relative 
strengths and weaknesses of each 

appreciate the importance of objectives in any 
assessment program 
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ASSBSSMBMTf MBASURBMBNT, TESTS AND EVALUATION: THE TERMINOLOGY 



Reading about assessment and evaluation can be very confusing 
because different authors use the terras in different ways. To 
avoid confusion we will settle for a particular set of 
definitions and avoid looking at alternative definitions- 
First let us consider four related terms— assessments 
measurement, tests and evaluation. 



Assessment 

Assessment is a most difficult term for educators. For some it 
means 'to test', for others it means 'to judge' or 'to evaluate'. 
Outside of education it can mean all three and additional things 
as well. 

But since this chapter is about educational assessment it is 
possible to narrow our definition somewhat. We can do this by 
reserving the term assessment for things to do with individual 
students. 

Assessment involves making judgments about students according to 
some set of rules or standards. 

Assessment can involve tests where marks are assigned according 
to some rules, but it can also be judgmental or based on opinions 
and feelings. 

The first type of assessment, the type based mainly on marks and 
rules is called OBJECTIVE ASSESSMENT. The second type, based on 
opinions and feelings is termed SUBJECTIVE ASSESSMENT. 

In practice an assessment is rarely purely objective or purely 
subjective. A true/false question on the surface seems highly 
objective. However it is important to remember that the decision 
to ask that question involved the opinion that it was a question 
that should be asked, and such opinions aie subjective. 

Consider the example of a girl who applies for a typing 
position with an advertising agency. 

She is given a typing test and is interviewed by the office 
manager. Her test results show that she can type at 42 
words per minute and makes no more than two errors per 
page — this is an objective assessment. 

The office manager looks at her work and sees that it is set 
out very well. But after the interview she reports that the 
girl was shy, seemed hesitant when answering questions and 
dressed in an unsuitable manner. 

The judgments regarding the netting outs shyness and 
appearance, if they simply reflect the manager's opinions 
and values, are all subjective assessments. 



It is, of course, possible to turn Subjective assessments into 
objective assessments. In the ubov^ case this would involve 
writing down checklists of standards for •setting out', 'levels 
of shyness' and 'appearance'. The applicants could then be 
graded using each checklist and producing an objective assessment 
of the three characteristics. 



Measurement 

Measurement is one way of making an assessment. Measurement is 
the assignment of numbers (marks) or grades according to some set 
0£ rules. 

For example, a mark of 9 out of 10 for an essay is a 
measurement; a comparable grade for such an essay would 
probably be an A on a possible range from A to E. 

Measurement may use a variety of instruments such as tests, 
rating scales and checklists. Each of these can be used to make 
measurements. 

For example, it is possible to measure the opinions of 
students as to the level of maintenance of their teaching 
areas using a simple 5 point scCi^Zb as follows: 

How do you rate the overall MAfSTENANCE of the various 
TEACHING AREAS (classrooms, laboratories, workshops)? Tick 
(y/) the box that corresponds with your opinion* 



VERI SATISFACTORY 


Q 


SATISFACTORY 


Q 


BORDERLINE 


Q 


UNSATISFACTORI 


Q 


VERY UNSATISFACTORY 


Q 



Should a large number of stude>it8 rate maintenance as 
'Borderline' to 'Very Unsatiefatstory' this is a clear 
measure of the level of stude>it dissatisfaction with 
maintenance . 



Tests 

A test is the most commonly used methocj of making a mea.surement 
in education. Tests can take a number of forms, some of which 
are described below. 

Achievement 

Achievement tests measure the amount Xearned after a period of 
instruction. 
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For example — after a period of instruction on tools used in 
woodwork^ a suitable achievement test to check the student's 
knowledge of tools might be something like the 'TOOLS 
CROSSWORD ' . 



TOOLS CROSSWORD 




CLUES 



Across 

3* A tool used when sharpening saws is called a saw 

6* A saw used for the cutting of masonite^ plywood etc. 

8* This chisel must not be hit with a mallet^ th€irefore it 
is used for fine clean up. 

12* This tool is made from a flat piece of tool steel 12Smm 
X 7Smm and is used for fine claan up of timber 
eurfaceo • 

IS* This tool has an adjustable cutter for boring holes up 
to 7Smmm 

%?• Used to turn Auger bits* 
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Across (cont) 

19. This group of tools reduces the timber surface by 
scoring or scratching it with a multiple of fine 
grooves . 

20* A curved chisel. 

21 . A marking tool used in transferring angles. 
Down 

1. This saw is used for cutting with the grain* 

2. This saw tak^s its name from a woodworking joint and is 
used for bench work. 

3. The name of a cramp used when gluing up frames on the 
bench. 

4. A tool used mainly for cutting internal curves where a 
bow saw cannot be used. 

6. Used when cutting timber. 

7, This tool can be either rough cutting or clean cutting, 
the most common being called a 'Jennings' type, 

9. This tool is used for quick shaping of curved surfaces. 

10. Used when marking out. 

11. Used to remove saw marks et:^. from rough sawn timber. 
The blade is slightly rounded. 

13. A tool used mainly for cutting scribes in beading and 
quadrant moulds. 

14. Two types of this tool are produced, the: a) Tower 

b) Lancashire 

and is used mainly for extracting nails or tacks which 
have become bent in driving. 

16* A hand tool used for cutting curves in thick timber. 

17. This tool is called a Finch <. 

18. Used in the clean up of curved surfaces . 

(From Basic Carpentry/ Joinery Trade: Basic Module 1.6, SA 
TAFE) 
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Aptitude 



Aptitude tests are designed to predict a person's future 
performance* 

Aptitude tests involve the solution of problems that are outside 
the student's direct experience. A test of knowledge cannot be 
an aptitude test. 

Aptitudes with 'number series' provide a suitable example. 

Although a student might be able to identify the rules for 
supplying the missing number from the series 

2, 4, ?, 25$ (Answer 16) 

another 'series' will require the identification of a new 
rule . So the series ' 

2, 4, ?, S, 4 (Answer 3) 

is^ different and can be assumed to be outside the student's 
direct experience . This classification works reasonably 
well if the test item involves something the student has not 
encountered or rarely encountered previously^ 

However, if the solution of 'number series' is a part of the 
curriculum and the student gets a lot of practice at solving 
such problems, 'number series' become achievement items 
rather than aptitude items. 

Many so-called spatial ability tests can be classified as 
aptitude testsr particularly if spatial ability is something not 
covered in the curriculum. 

For example.' 

If the shape shown in Figure I is cut out and folded, it 
forms a solid, the top half of which is shovn in Figure II. 





An edge common to the black face occurs with 

A Q only; c both Q and R; 

B R only; d neither Q nor R. 

(Answer C) 

(ACER MATHS ITEM BANK BOOK 2, 618011). 
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Diagnostic 



Diagnostic tests identify a student's difficulties or patterns of 
error. A diagnostic test is often used to provide the basis for 
remedial instruction. 

Fov example, in mathematics it is of fundamental importance 
to be able to recognise the symbols which represent the 
'four processes', addition, subtraction, multiplication and 
division* 

A series of diagnostic test items which address this issue 
might begin with: 

(Ql) 14 X 7 = ? (Q2) IS X 5 ^ ? (Q3) 16 t 4 = ? 

A 98 A3 A 64 

B 21 B 10 B 20 

C 7 C 20 C 12 

D 2 D 75 D 4 

Ard an answer pattern such as 

A, D, D, suggests there are no problems at this level with 

the symbols X and i 

whereas 

B, C, D suggests confusion between the symbols X and + 
and 

D, A, A suggests confusion between the symbols X and i 

Provided such error patterns are consistent over a 
sufficiently large number of items the teacher should be 
able to diagnose particular problems the student is having* 

Tests are further considered under the section on norm-referenced 
and criterion-referenced assessments pp 10-14. 



Evaluation 

Evaluation is the assessment of the merit or worth of something. 

Evaluations are about such things as programs^ institutions and 
policies. Assessments of students are typical components of an 
educational evaluation. Evaluation can be both 'quantitative* 
and *quali tative*. 

When an evaluation involves the consideration of numbers (e.g. 
the percentage of students passing a test) it is called 
'quantitative*. When it is based on opinions or values (without 
reference to numbers, grades etc.) it is called 'qualitative'. 

The quantitative information of evaluation consists of 
elements which have been assigned numbers, grades or ratings 
of some type. Measurements are quantitative information. 
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The qualitative inlormation used in evaluations usually 
consists of descriptive statements which aim to elaborate, 
clarify, and analyse the issues under consideration. 

There are two broad functions of evaluation which are categorised 
as formative and summative. 

Foraative evaluation 

Formative evaluation refers to the act of assessing the worth or 
providing information about something such as a teaching aid, a 
teaching program, a college etc. with a view to making on-going 
changes. Implicit in the process of formative evaluation is the 
requirement that the activities being evaluated can be modified 
during the course of the evaluation. Formative evaluations are 
characterised by flexibility and a commitment to making changes 
(improvements) as soon as these are perceived to be necessary. 

Summative evaluation 

Summative evaluation, on the other hand, assesses the worth or 
provides information on an activity at some predetermined point 
m time or on the activity's completion. Summative evaluations 
have little flexibility. They may be designed with improvement 
in mind, but any changes aimed at improving things are made at 
the end of the evaluation. 

Monitoring the use of an interim collection of curriculum 
materials in a classroom setting before they go for 
publication in a 'final form' is therefore the task of a 
formative evaluator. 

The formative evatuator's job is to ensure that changed are 
made that improve the materials prior to the production of 
the 'final form'. 

After these curriculum matt^rials have been enshrined between 
hard covers, a judgment about their worth would become the 
task of a summative evaluator. 



Bvaluation and research 

It is also useful to distinguish research from evaluation. 
Research studies are those primarily designed to add to the body 
of knowledge on the subject being researched. Research is about 
drawing conclusions; it emphasises the value of truth and has 
high generalisability. Evaluation, on the other hand, is about 
providing information for particular purposes and is less 
generalisable. 

An article entitled 'The theory of vocational education ' is 
likely to be about research in TAFE whereas the title 'How 
electronics ie taught in TAFE colleges ' suggests 
evaluation study. 
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ASSBSSMENT IN TAFB 

Assessment can be used for many purposes, but in the context of 
TAFB, there are two major functions namely: 

to assist in making decisions about the performance of 
individual students 

to provide information about the teaching and learning 
processes. 

The methods of assessment in a subject should be based upon the 
answers to two general questions. Firstly, for each set of 
objectives, what is the most valid, reliable and practicable 
method of testing? Secondly, how can this method of testing be 
most appropriately conducted? 

In considering these questions the following points need to be 
taken into account. 

Bnd*of*year examinations are, in many cases, an appropriate 
means of testing knowledge and understanding of the content 
of a subject and, to some extent, how this knowledge and 
understanding can be applied. But such examinations do not 
allow suitable feedback to the teacher or the student. Bnd- 
of-year examinations are summative assessments of student 
performance. Since they come at the end of a period of 
instruction there is no chance for their results to be used 
for any on-going improveroent of student performance. Of 
course, the teacher might be able to reflect on the results 
and do something to improve his or her teaching next time. 
Unfortunately, this is too late for the group of students 
who have just been tested. 

Conversely, well-designed class tests can readily provide 
the required feedback. Results of class tests, if used for 
improvement in the teaching and learning situation, are 
formative assessments. 

In many TAFB subjects application of knowledge and 
understanding is most validly tested by a task that 
replicates, as far as possible, the si tua "-ion in the 
industrial or commercial field for which the subject was 
designed to prepare the student. 

Generally, TAFB students are more positively motivated by 
tests relevant to on-the-job performance and by continuous 
assessment than by tests of knowledge and understanding or 
by end-of-year examinations. (Continuous assessment 
involves testing the progress of students in a subject at 
regular intervals, say for example, on the completion of a 
unit of work. In some ways the term is an unfortunate one 
as it is taken [quite wrongly ] by some people to mean 
continual assessment.) 




Continuous assessment is better able to provide feedback on 
the effectiveness of the educative process to teachers and 
students than is e^/id-of-year assessment. 

After making such considerations/ the specific objectives of a 
subject can be assigned to various methods of assessment such as 
end-of-year examinations, class tests, assignments, projects, 
practical tests, orals and aural tests. These various methods of 
assessment are considered in greater detail in Chapter m. 

Two general rules emerge from the above: 

1 • Continuoue aeeeeement ie preferable to once only end- 
of-year examinations because it 

gives a greater motivation to students 

provides regular feedback on performance to both 
teachers and students, 

2. Real-life practical tests are preferable to artificial 
ones . 



The role of norm^refpirenced and criter ion^referenced assessments 

The following anecdote outlines the difference between these 
types of assessment: 

Dr Wun after examining a patient who had been in a road 
accident was asked by the patient's family what was the 
chance of recovery, Dr Wun shook her head and replied, 
'Well, her kidneys have been badly damaged and we 
haven't been able to stop the internal bleeding* I'm 
afraid there is little chance for her'. 

Dissatisfied with this prediction the family sought a 
second opinion from Dr Too. When asked the same 
question she smiled reassuringly and said, 'Well she is 
very ^ low, but compared to the other women in the 
hospital ward her chances of recovery are quite good'* 

In making^ a judgment about the patient's chance of 
recovery in relation to the criteria of kidney damage 
and internal bleeding, Dr Wun was making a criterion- 
referenced assessment * ' ' 

Dr Too's judgment, based on the chances of survival 
compared with that of other patients in the ward, can 
be likened to a norm-referenced assessment* 

It is a moot point as to whether the two diagnoses are 
assessments or evaluations but assessment is probably the 
preferable word, since we said at the beginning of the chapter 
that the term would be reserved for things to do with 
individuals. Irrespective of this, the distinction between norm- 
referenced and criterion-referenced holds. 

180 



Norm-referenced assessment 

In a norm-referenced assessment a test is used to find the 
pe::formance of an individual on a defined set of tasks in 
relation to the performance of others who complete the same test^ 

For example, a test in fitting and machining that consisted 
of twenty short answer questions which are known to vary in 
difficulty from easy to hard, when used to compare the 
performances of students in a class, would be a norm^ 
referenced test. 

Criterion-referenced assessment 

In a criterion-referenced assessment a test is used to find the 
performance of an individual in relation to some specified 
standard. 

For example, a test of spelling for which the teacher 
announced that she required 9 correct answers out of 10 in 
order to 'pass' would he a criterion^veferenced assessmentm 
(The standard being 9 out of 10). 

Using norm-referenced and criterim-referenced assessment 

Norm-referenced assessment provides a comparison with other 
individualSf criterion-referenced assessment provides a 
comparison between the individual and some previously defined 
objective or set of objectives. 

Tests used in norm-referenced assessment usually produce a range 
of scores, that is, they tend to spread the students out. On the 
other hand, cr iter ion-referenced test scores usually clump 
students together, the point being that spreading students over a 
range is not essential to a criterion-referenced test. 

A criterion-referenced test is analogous to a high jump in which 
the high jump bar is fixed at 1 metre. Everybody taking the test 
is either a successful jumper or unsuccessful jumper. 

Note: it does not matter by how much a test taker clears that 
bar — the task is simply to get over (i.e. to pass the 
test) . 




Compartso^is bet ween norm-ref erenced and cr iterion-refer eticed 
assessments — — 



The following table contrasts some of the more imoortant aspects 
of the two types of assessment. 





HORN-RBFBRSNCBD 


CRITERION'RBFBRBNCBO 


USES 


To provide a means of 
comparing the performance 
of individuals on a set of 
defined tasks. 

As a selection device to 
select between individuals 
competing for scarce 
resources (e.g. places in 
an engineering school). 


To determine the peforraance 
of individuals in relation to 
specified criteria 
(objectives) . 

As a 'hurdle' to decide 
whether an individual has 
gained enough knowledge 
and/or skills (i.e. achieved 
sufficient mastery) to 
proceed vo the next level of 
instruction. 


CONTENT 

1 


Content can be defined 
by general objectives. 

Test content does not 
necessarily match part- 
icular classroom object- 
ives closely. 

Relationship between test 
items and classroom goals 
may be indirect. 

Test items are normally a 
representative sample from 
a larger poolf the assump- 
tion belag that perform- 
ance on the sample is an 
accurate indication of 
performance on the whole 
pool. 


Content must be defined by 
specific (behavioural) 
objectives. Furthermore ^ in 
many TAPE subjects, perform- 
ance criteria and conditions 
under which performance must 
be accomplished also need to 
be specified. 

Test content must match 
classroom objectives closely. 

Relationship between test 
items and classroom goals 
must be direct. 

All ntated objectives need to 
be tested by items; sampling 
of objectives is not 
appropriate. 


QUESTION 
TYPE 


Items which discriminate 
between individuals are 
desirable as there is 
usually a requirement to 
compare the performance 
of /different students. 


Items must match the object- 
ives; items which discrimin- 
ate between more able and 
less able students are not 
sought. Items are required 
to provide information about 
what a student can or cannot 
do. 
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NORM-RBFBRBNCBD 


CRITBRION-RBFERBNCED 


SCORES 


Variability in scores 
obtained by the group of 
students taking test is 
desirable. 

Scores provide information 
on relative standing of 
student in group and give 
indication of what student 
can do. High scores can 
reflect excellence of 
Der f omance > 


Variability in scores is 
irrelevant. 

Scores reflect accurately 
what students can or cannot 
do. Identifying excellent 
performance is often not 
appropr iate . However , as 
with norm-referenced tests, 
it is usually poss i ble to 
identify performance well 
above the specified standard 
or criteria. 


GRADES 
AND 
CRI- 
TERIA 
FOR 

SUCCESS 


Variability of scores en- 
ables allocation of 
grades. 

Scores (in some tests) can 
be related to performance 
of larger group of 3ta* 
dents giving normative 
data, as in a standardised 
test. 

e.g. 'He came top of the 
class* 

'Sis score puts him in the 
90th percentile of the 
population * 

Different criteria for 
success are possible, 
e.g. 5 0% may be classified 
as a pass but students may 
be required to score 75% 
if they are to proceed to 
next level of study. 
(N.B. such criteria are 
arbitrary) 


Grades, except pass/fail, may 
not be appropriate. 

Scores usually expressed in 
such a way as to reflect 
performance against criterion 
e.g. 'She can type 40 words 
per minute without making 
nore than 2 errors per page ' . 

Criteria for success are 
often matters for judgment. 
In many TAFE subjects such 
judgments are usually related 
to occupational performance 
data. In the above exaunple 
more than 2 errors per page 
might be defined as failure 
to achieve the criteria. 


RELIA- 
BILITY 
AND 
VALID- 
ITY 


A well-developed set of 
psychome tr ic techniques is 
available for deciding on 
the reliability and valid- 
ity of the norm-referenced 
test. 


Test theorists are still in 
the process of developing 
generally accepted statistic- 
al procedures for determining 
the reliability and validity 
of the criterion-referenced 
tests. Some of the 
procedures currently in use 
involve complex 
mathematical formulae. 



.{After Boehm (1973) ] 
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The future of no rn* referenced tests and cr i ter ion-re ferenced 
testa ~ 



Both norm-referenced and criterion-referenced tests have their 
problems^ but it seems unlikely that one or the other will 
disappear from the education scene in the future. 

Advocates of the two test forms often fail to acknowledge the 
imperfections of their tests. This has sometimes made it 
difficult to decide which type of test is the better or more 
appropriate. For instance^ although few would deny the value of 
specifying tne objectives of teaching, arguments do start when 
try to decide how specific the objectives should be. 

The • technical education' sector of TAFE has long been associated 
with very specific objectives and criterion-referenced testing. 
Indeed, this association pre-dates the advent of the term 
•criterion-referenced' into the literature on testing. 

Whenever the purpose of assessment is to measure the performance 
of an individiual on a job-related task, a criterion-referenced 
test IS called for. Once the performance or task to be 
accomplished is identified, the criterion-referenced test 
developers construct their test items using a knowledge of the 
standard of performance required and the cor>ditions under which 
that performance must be accomplished. These points are taken up 
in more detail in the next section. 

Norm-referenced testing has more relevance when issues of 
selection are involved. If the task is to select those 
individuals in a group who are most liK.ely to succeed in a 
course, and there are few places available, then criterion 
referenced testing is of limited value- 

For example^ if 20 etudente have met the basic criteria 
(i.e. paeeed the criterion^-referenced test) hut there are 
only 5 places available for the 20^ selection is a problem. 
In such cases a norm^referenced test which spreads the 
students out over a range of scores can make selection 
eajzer — places are simply offered to the individuals with 
the fzve highest scores. 

Sometimes it is possible to modify criterion-referenced testing 
procedures to have a base criterion (a pass) and progressively 
graded performances above the basic criterion. 

An example could be machining a rod to a particular set of 
tolerances and having three progressively more demanding 
sets of tolerances for which the student could be rewarded. 
This uould be an acceptable way of distinguishing between 
our 20 students competing for the 5 places; however it is 
important to realise that we are no longer dealing with a 
pure cfiterion-referenced test situation. The act of 
spreading the students over a range has produced a hybrid 
criter,. vn^referenced and norm^referenced test. 
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ERIC 



OBJBCTIVBS 



Objectives are the subject o£ the next chapter and will not be 
dealt with in great detail at this stage. However, objectives 
need to be mentioned in an introductory chapter since they are 
the starting point of all tests. 

Cr iter ion->ref erenced testing requires that the objectives of 
instruction be clearly defined in observable behavior.ral terms. 

A general objective like 'understands the meaning of the term 
scientific hypothesis* is an unsuitable basis for a criterion- 
referenced test although a norm*referenced test could use such an 
objective. 

Specific (behavioural) objectives define the content of the 
criterion-referenced test but ther^ is still a further 
requirement and that is a definition of minimal acceptable 
performance. 



Miniaal acceptable performance and level of specificity in 
criterion-referenced tests 

For most purposes it is enough to specify how well the learner 
must perform for the behaviour to be acceptable. 

This normally involves 

Identifying the outcome required (end behaviour) and 

indicating how well the learner must per form for the 
behav iour to be acceptable (cr i ter ion) . 

Far example^ 'can type 40 words per minute without making 
more than 2 errors per page'm 

However, some people insist on a much greater degree of precision 
and require the specifications to also: 

state the conditions under which behaviour is to occur by: 

indicating what materials, if any, will be provided to 
the learner 

stipulating the time limits within which the learner 
must complete the task. 

Consider an example of such a precise specif ication. 

Students will he able to construct and measure to an 
accuracy of 1 millimetre a perpendicular line from the apex 
to the base of a triangle in at least four out of five 
examples presented within a 30 minute period, using only a 
ruler, compasB and pencil. 
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Tightly prescriptive objectives like t.*e above have drawn their 
share of criticism mainly because of the constraints they place 
on teachers. 

Critics point out that levels of performance and time limitations 
are sometimes arbitrasry and claim that teaching based on such a 
detailed set of requirements is over-formalr rigid and dull. 
However such criticisms are usually 'course specific'. in some 
Ti\PE courses, meeting these specific conditions is the challenge 
of the learning process. A TAFE student's success in the world 
of work depends often upon how accurately and how efficiently he 
or she performs certain tasks. These points are taken up again 
in the chapter en Objectives. 
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Boehntf A* E. Criterion-referenced assessment for the teacher. 
Teachers College Record r 1973^ 75(l)r n7-126. 

Mathematics item bank, ACER, 1978. 
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K XBRCISB I (1^ 
DIAG«OSTIC AMD ACHIBVBMBNT TESTS 



Complete the following tnble by writing in the empty boxes the 
appropriate comparison between a diagnostic and a norm-referenced 
achievement test. 

Yo\ can check your answers by turning to page 20. 



■ .1 1 
CHARACTBRISTIC 


DIAGNOSTIC 


NOim-REFBRBNCED 
ACHIEVEMENT 


Purpose 


To identify a student *s 
specific learning 
difficulties. 




Ideal spread 
of marks 




Sp ead across a range of 
scores, from high to low 
achievement. 


Importance of 
time limits 




Depends on test specifi- 
cations but it is usual 
for the limit to be 
stipulated. 


Hazards 


Inappropriate use of a 
student's total mark. 




Uses 




Selection and 
prediction. 
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BXERCISB I (2) 
KORK-ANP CRITBRIOW-RBFBRBHCBD TESTS 



Below are a number of points about norm-referenced and criterion- 
referenced tests. 

Complete the table by writing into the empty boxes the contrasting 
points. 

You can check your answers by turning to page 21. 



CHARACTERISTICS 


NORH-REPBRENCED TEST 


CRITERION-REFERENCED TEST 


USE 




As a 'hurdle' to decide 
whether an individual has 
gained enough knowledge 
and/or skills (i.e. 
aicn le V eu sur r icien u 
mastery) to proceed to 
the next level of 
instruction. 


CONTENT 


Content can be defined 
by general objectives. 




SCORES 


Scores provide informa- 
tion on relative stand- 
ing of student in group 
and give indication of 
what student ran do. 
High scores c'.n reflect 
excellence ol perform- 
ance. 




GRADES 


Variability of scores 
enables allocation of 
grades. 




QUESTION 
TYPE 




Items must match the 
objectivesr elisor iminat- 
ing items are not sought. 
Items are required to 
provide information about 
what a student can or 
cannot do. 
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SUGGESTED AHSWBRS 



EXERCISE I (1) 



CH/JRACCBRISTIC 
OF TEST 


DIAGNOSTIC 


NORM-REFERENCED 
ACHIEVEMENT 


Purpose 


To identify a student's 
specific learning 
difficulties. 


To measure the degree to 
which a particular set 
of abilities is present 
in an individual. 
Measured against other 
individuals) . 


Ideal spread 
of marks 


Not relevant. 


Spread aciross a range of 
scores, from high to low 
achievement. 


Importance of 
time limits 


Must be aenerous hn t 
slowness itself can be 
a specific difficulty. 


cations but it is usual 
for limit to be 
stipulated. 


Hazards 


Inappropriate use of a 
student's total mark. 


Decisions regarding 
achievement dependent 
upon comprehensiveness 
and quality of test 
i terns . 

Since comparisons are 
being made between 
individuals there can be 
no absolute standard. 


Uses 


Identification of areas 
in need of remediation. 
Making decisions about 
individual instruction. 


Selection and 
prediction. 
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SOGGBSTBD ANSWERS 
BXERCISB I (2) 



CHARACTERISTICS 


HORM-RRFERENCED TEST 


CRITERICm-REFERBNCBD TEST 


USES 


As a selection device 
to select among indivi- 
duals competing for 
scarce resources (e.g. 
places in a medical 
school) « As a means of 
comparing an individual 
against the performance 
of a population (e.g. 
X • Ve* uesu j . 


As a 'hurdle' to decide 
whether an individual has 
gained enough knowledge 
and/or skills (i«e. 
achieved sufficient 
mastery to proceed to the 
next level of 
instruction) . 


CONTENT 


Content can be defined 
by general objectives. 


Content must be defined 
by specific (behavioural) 
obj ec tives . 


SCORES 


Scores provide informa- 
tion on relative stand- 
ing of student in group 
and give indication of 
what student can do. 
High scores can reflect 

ance. 


Scores reflect accurately 
what students can or can- 
not do. Identifying 
excellent performance is 
often not appropriate. 


GRADES 


Variability of scores 
leads to allocation of 
grades. 


Grades, except pass/fail, 
may not be appropriate. 


QUESTION 
TYPE 


Items which discriminate 
between individuals are 
desirable as there is 
usually a requirement to 
compare the performance 
of different students. 


Items must match the 
objectives, discriminat- 
ing items are not sought. 
Items are required to 
provide information about 
what a student can or 
cannot do. 



21 



23 



CHAITKR II 



OBJBCTIVBS 



WHAT THIS CHAPTSR IS ABOUT 

This chapter explains the importance of ot^jectives in student 
assessment. 

An introduction is given to various types of educational 
objectives. 

Examples are given of the relationship between the educational 
objecti^ve and the test item which is written to assess that 
objective. 



OBJECTIVES 

After completing this chapter, you should be able to: 

explain the meaning of the term 'educational 
objective ' 

know that there are three types of objectives-^ 
cognitive (thinking), affective (feeling), and 
psychomotor (doing); and recognise each type 

clasBr.fy particular test items in terms of the 
educational objectives they are measuring 

understand that objectives have limitations in 
some areas of education and explain why this is 
so. 



INTRODOCTION 

An educational objective is a planned outcome which results from 
some process of learning. A 'planned outcome' could be the 
ability to measure the diameter of a piece of wire using a 
micrometer screw gauge. The 'process of learning' which makes 
this possible could involve the student receiving instruction 
from a teacher and then practising the skill with various pieces 
of wire. 

However r some educators would see a problem wi th the above 
example in that its level of specificity (detail) is in doubt. 
For instance, we do not know how accurate the student must be in 
order to be judged as having mastered the objective. 
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Educational objectives can, in fact, range from broad statements 
(called goals o-: aims) to very precise statements, called 
behavioural or specific objectives. Behavioural objectives are 
so called because they specify the performance (behaviour) the 
student needs to display to satisfy the objectives. 

The levels at which objectives exist can be seen from the 
following example. 

TO APPRECIATE THE IMPORTANCE 
OF SCIENCE AND TECHNOLOGY TO 
SOCIETY 

whereas 

TO UNDERSTAND THE MEANING AND 
USE OF SCIENTIFIC HYPOTHESES 



but 

TO DEFINE THE MEANING OF THE 
TERM •SCIENTIFIC HYPOTHESIS' 

and 

TO DISTINGUISH BETWEEN CORREC 
AND INCORRECT SCIENTIFIC 
HYPOTHESES 

Some educators increase the level of specificity even more by 
adding the conditions under which the behaviour is to occur, and 
the minimally acceptable perfocaance levels as well* 

For example, in Chapter I a highly specific objective was 
quoted as: 

Students will be able to construct and measure to 
an accuracy of 1 millimetre, a perpendicular line 
from the apex to the base of a triangle, in at 
least four out of five examples presented within a 
30 minute period, using only a ruler, compass and 
pencil* 

As objectives become more and more specific they begin to read 
like actual tests. Indeed, some authors refer to 'performance 
objectives* when describing any objective which specifies: 

a skill to be performed 

the conditions under which it must be performed 

the acceptable standard of performance to be achieved. 

Such an approach is relevant to man^ areas of TAFE, especieiily 
when the aim is to relate a performance test to the real world. 



is a very broad objective of 
education, which is better 
called an AIM or GOAL 



while still a broad statement, 
is in a subset of the above 
arJ can be called a GENERAL 
OBJECTIVE 



are two exeuaples of 
BEHAVIOURAL OBJECTIVES arising 
from the general objective 
above. (Note that they 
specify what the student must 
r 1 be aLIe to do) • 
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The example of measuring the diameter of wire can be turne;^ into 
a performance objective by prescribing the acceptable tolerance 
limits within which the measurement must be made« 

PERFORMANCE OBJECTIVE 

Skill measure the diameter of a copper wire 

Conditions given a length of wire and micrometer screw 

gauge 

Standard to an accuracy of + or 0*005 mm. 



CEASSIFICATION OF OBJECTIVES 

There are a number of systems for class if ying educational 
objectiveSi^ however, this Chapter will consider only the scheme 
devised by Bloom and his co-workers (1956). 

This system has its critics and is not being presented here 
as an 'exemplary method of classification. Zt is, however^ 
the best known and mo^t researched of all the methods and a 
major advantage is that over the years it has be&n used to 
classify many thousands of test items. 

TAFE teachers should see it as a useful method of 
classification but should feel free to modify, iv as 
necessary to meet their needs. 

The system developed by Bloom and his co-workers is called a 
tsxonomy because it is a system of classification. The taxonomy 
t:lassifie5 within three basic ared^s or domains, namely: 

cognitive 

affective 

psychomotor. 

The cognitive domain includes objectiveb which deal » ith the 
recall or recognition of knowledge and the development of 
intellectual abilities and skills. 

The affective doaain objectives describe changes in interests, 
attitudes and values, and the development of appreciation* 

7he psychomotor domain deals with manipulative (motor) skills. 

These three domains are sonaetim^^s referred to as the 3H^s; Head 
(cognitive). Heart (affective) and Hands (psychomotor) to help 
emphasise the distinctions. 
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COGNITIVE 
AFFECTIVE 
PSYCHOMOTOR 



THE HEAD 
THE HEART 
THE HANDS 



THINKING 

FEELING 

DOING. 



Each domain is broken down into a number of levels as follows: 
Cognitive Domain 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 



levels of cognitive 
lear.iing 



Affective Domain 
Receiving 
Responding 
Valuing 
Organisation 
Charac terisation 



levels of affective 
learning 



PsychOiiiotor Domain 

Reflex movements 
Basic-* fundamental move^nents 
Perceptual abilities 
Physical abilities 
Skilled movements 
Nondiscursive communication 



levels of psychomotor 
learning 
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A brief outline of the COGNITIVE DOMAIN levels: 

i. Knowledge^the recall of specific or general information, 
patterns, structures, or settings. 

The ability to recall the names of different tools 
involves ' knowledge * . 

ii. Coroprehension~the level of understanding at which a.i 
individual knows what is being communicated ard can use th s 
knowledge in some way. 

There are actually three types of comprehension. 

The first involves translation, that is, putting a 
communication into another language. 

Reading a graph involves translation . 

The second type is interpicetation. 

Reading a paragraph and hning able to identify and 
comprehend the major ideas involves 
interpretation . 

Bztrapolation is the third type. 

Predicting a trend based on a comprehension of the 
information provided involves extrapolation * 

iii. Application~>the use of abstractions in particular and 
concrete instances. 

The ability to predict th^ probable result of 
introducing a change such as substituting one 
metal for another in the manufacture of batteries 
involves ' application '* 

iv. Analysis— the breaking down of a communication into its 
component par ts and understanding oi the relationship 
between these parts. 

The ability to read a report and distinguish fact 
from opinion requires ^ analysi s'. 

V. Synthesis—- the arrangement aivj combination of pieceSi^ parts, 
or elements into a coherent pattern or structure.^ 

The skill in writing a report that requires the 
organisation and development of ideas inv ylves 
' synthesis ' . 

vi. Evaluation— the quantitative and qualitative value judgments 
on factors such as materials, methods, the use of a standard 
of appraisal, antd uhe utilisation of criteria. 

The ability to compare tioo conflicting ways of 
carrying out a project and judge the advantages 
aud disadvantages of each requires ' evatuatiori '* 

Bloom et al (1956) can be consulted for a more detailed breakdown 
of the levels i to vi. 
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Cognitive dcHnain examples 



The exampios given below are meant to help the reader understand 
the Bloom taxonomy- It should be noted that classifying items, 
other than knowledge items, in this way assum*5S the particular 
answer is not arrived at by recalling something already taught. 
What can involve 'comprehension' or 'applica tion' for some 
students may be •knowledge* for others. 



Knowledge 

Kl The tool shown in the diagram is 

A a stake 

B a boseing mallet 

C a pinch bar 

D an anvil 




(Answer C) 

This item tests knowledge of a specific piece of information, 
that is, knowing the name of a particular tool. 

CoBpr ehen s ion 

For questions Cl and C2 consider the diagram which describes the 
process of fibre board manufacturing. 



PUIP PftCPAfUnON 
I SUPREME 

I Pinelog8M« 
ddbvenidloMff 



i Locsar* 
tinitMrpii 



reduced to 



M Tuntw pieces 
flfB reduced 
tofibrouspdp 

bt Refinert produce 
fine uniromi puip 



n SAJWSON 

< Mineral Abre received 
at (aclofy in belee 

« Baies ere reduce 
toicoeefibree 



tod into 



iv Additives 
fomwiga 



weigheda 

mixing tan 



mjied wtth 
water to toon 
a slurry 




&iCttoo«lBf ii Press follt iii Wit mat is cut 

tom» a conttnuous remove moisture to tenmh and 

-emat to determined driedinamuttt- 

thtchness deck <>vcn 
and density 



F<MMING 

Hi 




i Board is dimmed 
to size and ii Product is packed 
fordetiverv 



various surface 
fintahosara 
applied to meet 

Invvfcettng 
reguirements 
" FlNtSHlNG 

rv 



th^')ughout 
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CI Each stage in the process of manufacturing Supreme and 
Samson boards has been identified by a number* For 
example, I ii is the stage at which the 'Logs are 
reduced to timber pieces'* 

Which of the following most accurately identifies the 
stages at which water is removed from the material 
being processed? 

A I ii, II iv. III ii. III Hi, IV i 

B II iv. III ii. III Hi, IV i 

C II iv. III ii. III Hi 

D III ii. III Hi* 

(Answer D) 

This item requires the identification and translation of 
information presented in diagrammatic form. It is an example of 
the Translation level of a Comprehension question. 

C2 yhich of the follov^ing best describes the processes of 
manufacturing Supreme and Samson board? 

The processes for manufacturing Supreme and Samson 

A are similar in most respects 

B are simitar but differ mainly in the pulp 
preparation stage 

C are similar but differ mainly in the 
fin Lshing stage 

D are different in most respects . 

(Answer B) 

This item requires the interpretation of a communication given in 
the form of a diagram. It is an example of the Interpretation 
level of a Comprehension question. 
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Application 

Teachers and curriculum (designers have long recognised that 
students do not veally ' under stand' an idea or principle unless 
they can apply it in a new problem situation. Success with 
•application" questions, therefore, implies the student has 
understood the topic concerned. 

Consider the following item: 

APPl A straight stick half in and half out of a bowl of 
water appears bent because 

A light rays do not travel in a straight line 
in water 

B water droplets 'bend' the light rays 

C light rays change direction slightly when 
thry go from water to air 

D light rays scatter when they enter water. 

(Answer C) 

This item requires the student to explain a new phenomenon — the 
apparent bending of the stick — in terms of a known principle, 
that is, the diffraction of light. 

Note: 

In order to classify APPl as an 'application* question it is 
necessary to assume the student has not been taught the 
answer, if the reason for the stick appearing bent has been 
taught then this becomes a 'knowledge' question. 
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For APP2 consider the following diagram of a sewerage system. 



UPSTREAM 
VENT 



SOIL 
PIPES 



LJ 



WASTE 
PIPES 



HOUSE DRAIN 



v. 



1^ I 1^ RETICULATION OR BRANCH MAIN SEWER II ^ 



III 



TO TREATMENT 
PLANT 



MAIN SEWER 



SEWERAGE SYSTEM 



i4PP2 If faults or bl oekages in the ayatem can occur at 
points I, II and III, which of these is the 
responsibility of a plumber emploj^ed by a householder 

A all of I, II and III 

B I ^nd II only 

C I only 

D none of I, II or III* 

(Answer D) 

This item requires che application of a general rule or 
procedure; it is also arguably an 'interpretation' question, 
however, the rule of thumb for classification is to accept the 
higher level. 
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APP3 In the following diagram^ if the length of PQ is 3 
metres and the roof slope is 21^ 80', what is the 
length of the half span? 




greater than Va rafter thickness 



COMMON RAFTER 



This item tests the ability to apply the laws of trigonometry to 
a practical situation. 
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APP4 Enter the following tvaneactione into the journal and 
ledgers (general and aubaidiary) of J. Chaaton. 
Balance the account a, prepare a trial balance and 
debtora' and creditora' achedulea at 31 May 1984. 

J. CH ASTON 

1984 
May 

1 Commenced buainea a with the following aaaeta and 
liabilitiea : 

Debto ra $ 7 0 0 J Building a $ 4 j 5 0 0 j Stock $5 4 0 j Bill a 
receivable $400; Office furniture $370; Creditora $580; 
Bank overdraft $800; Billa payable $100 
Debtora - W. Hall $250, W. Lofta $175, J. Gill $275 
Creditcra - G. Collia $320, C. Stewart $260 

3 Caah aalea $140 (CRS) 

Credit purchaae from B. Gill $300 (Inv,91) 

5 Gill charged $10 cartage on purchaae on Invoice 91 
Paid advertising $60 (Ch. 31) 

7 Caeh aalea $640 (CRS) 

Loan from /?. Crowe $1, 000 - Received 

9 Credit aalea to J* Miller $250 (Inv, 21) and B* Cox 
$245 (InV.22) 

Proprietor drew caah for own uae $5 0 (Ch, 32) 

1 0 Gooda returned to ua by Miller $25 ( Correction note 39) 
Caah aale to G* Reveruzzi $440 

Paid inaurance $75 (Ch, 33) 
15 Credit purchaaea f rom C ollia $10 0 ( Inv m 2 7) and^ P« 

Barry $85 (Inv. 31) 
18 c7« Gill paid ua $143 and we allowed him $7 diacount 

(Bee. 41) 

Paid office expenaea $55 (Ch. 34) 

We returned gooda to Collia $15 (C/H 40) and Barry $10 
(C/U 71) 

23 Bought furniture for office from J. Bennett for $90 
caah (Ch. 35) 

Credit aale to J. Gill $70 (Inv. 23) and J. Goold $95 
(Inv. 24) 

25 Allowance to Goold for overcharge $15 (C/U 40) 
Paid intereat on loan for May $5 (Ch. 36) 
Hall declared bankrupt and can pay only 50 centa in ^- 
balance written off (Rec. 42) 

28 Caah purchaa 3a from J. Greenwood $80 (Ch. 37) and S. 
Silvan $200 iCh. 38) 
Caah ealea $475 (CRS) 

30 Paid Collia $275 and he allowed ua $15 diacount (Ch. 
39) 

Cox aettled hi a debt in full by paying ua $13 8 (Rec. 
43) 

This question requires the application of principles of 
accounting to a new or unique situation. 
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Analysis 



Analysis emphasises the breakdown of information into its 
constituent parts and the detection of the relationships of the 
parts and the way they are organised. 

ANl Light has the characteristics of a bullet and a wave. 
Which one of the following is evidence of the wave 
characteristics of light? 

A light can be reflected by a mirror 

B a beam of light spreads out after passing 
through a small opening 

C light causes a current to flow in a photo^ 
electric cell 

D light carries energy 

(Answer B) 

This item requires the student to infer a relationship between a 
physical proper ty— wave motion—and a condition that must 
result— diffraction. 
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Question AN2 refers to the following information 
CLASS C FIRES 

CLASS C fives involve live electrical equipment such as 
light fitting 8, motors, generators, cables, wiring, 
switches, switchboards and electronic equipment. 

Blanketing, or smothering the fire to exclude oxygen is the 
most effective way of extinguishing Class C fires. 

Vapourising liquid (Bromochlorodifluoromethane, BCF), dry 
powder ^ chemical and carbon dioxide (CO^) extinguishers are 
effective in putting out Class C fires. 



f/ARNING : 

The vapourising liquid gives off toxic gases 
when in contact with fire. Take care with 
such equipment when used in confined spaces. 
Wear proper equipment, or ensure proper 
ventilation . 

The extinguishing agent used must be a non- 
conductor of electricity in order to avoid 
shock and further damage to equipment. 

never use water, or foam extinguishers, on 
Class C fires. 



AN2 Assuming you have suddenly come upon a fire which is 
burning strongly in a large switchboard attached to the 
wall in the switchboard room of a block of offices. 

You have at your disposal only four items: 

I a fire blanket 

II a BCF extinguisher 

III a CO 2 extinguisher 

IV a foam extinguisher . 

Based on the information provided, which of the 
following would be the most appropriate to extinguish 
the fire? 

A open doors and windows and use I 

B close doors and windows and use II 

C open doors and windows and use II or III 

D close doors and windows and use IV. 

(Answer C) 

This item requires the student to understand and analyse the 
interrelationsttips between the points made in the information 
supplied. 
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Synthesis 



Synthesis is the putting together of parts to make a whole. 

Most essay questions fall into this category — it is rare for an 
objective multiple choice question to qualify for the synthesis 
level. 

51 Compare and contrast the first aid procedures you would 
use in treating a person who had been badly burnt on 
the arm by hot acid and was at tne same time 
unconscious as a result of inhaling the acid fumes. 

This item tests the ability of the students to develop a plan of 
action to satisfy two sets of conditions. 

52 Below are partially completed drawing s of a split 
bearing house. 

Referring to the sectional side view and shaft detail, 
and to data on relevant parte, complete the drawings by 
adding a third view showing all internal details of the 
housing only. 

Completely dimension the drawing. Label third view and 
show any missing details, dimensions, cutting planes 
and surface texture symbols where required. 
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This item requires the development of a plan to satisfy a set of 
prerequisite conditions. 



Evaluation 



Evaluation is about making judgments. I t can involve some 
combination of all the other behaviours of Know ledge, 
Comprehensionr Application, Ana lysis and Synthesis. What is 
added are criteria involving values. Evaluation is a link 
between the cognitive and affective domains but with the emphasis 
on the cognitive. 
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It may be worthwhile for the manufacturer of bueineee 
goods to use space in magazines specialising in current 
affairs in order to have the opportunity to reach those 
readers who as buflinees people are i'lterested in buying 
the manufacturer' s product. Consider each of the 
following business goods. Tick (^) box A o^ B opposite 
each product as follows: 

A if such an approach would appear to provide enough 
worthwhile coverage to justify the expenditure; 

B if such an approach would not be likely to reach 
enough good prospects to Justify the expenditure. 



A B 

Holden car Q Q 

National rock drills ( mining industry) Q Q 

IBM typewriters Q Q 

Buckner tfzxtile machines Q []] 

Xerox Bond Paper. Q Q 

This item involves judging an approach to advertising in terms of 
a particular criterion. 



E2 4 home owner has a room with a stained pine floor which 
is in good condition. She needs to decide between two 
floor treatment s^^polishing with a floor wax o\* 
painting with a clear epoxy resin. 

For each of the qualities listed lelow check {^) which 
treatment is superior. If there is no difference 
between treatments do not check a box. 

Quality 



resistance to scratching 

resistance to sunlight 

overall appearance 

safety when walking over 
surface 



TREATMENT 
Polished Painted 

□ □ 

□ □ 

□ □ 

□ □ 



frequency of treatments rn i— i 

over year ' J 

cheapness of treatment Q Q 

This item involves making a number of judgments against a set of 
given criteria (qualities). 




The two sections which follow introduce the different sub- 
divisions of the Affective and Psychomotor Domains. No items are 
presided in these sections. Chapter III, however, contains a 
number of items from these two domains. 



Affective Domain 



The Sab-divisions of the affective domain are: 



i. Receiving — the sensitivity of the learner to the 
existence of certain stimuli. This level of affective 
learning includes three categories: 

(a) awareness 

(b) willingness to receive 

(c) controlled or selected attention. 

ii . Responding — the active response of the student to 
stimuli. This level of affective learning has three 
sub-ca tegor ies : 

(a) acquiescence to respond 

(b) willingness to respond 

(c) satisfaction in response. 

iii. Valuing— the development of an attitude toward a 
stimulus by the student. The prrc'cess of valuing is 
further divided into: 

(a) acceptance of value 

(b) preference for a value 

(c) commitment to a value. 



iv. Organisation — the level of affective learnir.g at which 
the student begins to build a value system. This 
category is subdivided into: 

(a) conceptualisation of a value 

(b) organisation of a value system. 



V. Characterisation of a Student's Value Complex — the 
organisation of a student's values into some kind of 
internally consistent system. This level of affective 
learning is divided into: 

(a) the development of a generalised set 

(b) the characterisation of a value system having as 
its object all that is known or knowable. 
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Paychc»otor doaain 



The subdivisions of the psychomotor domain are: 

i. Reflex movements~involuntary movements of the body 
which are functional at birth and develop with the 
maturation of the individual. 

ii. Basic-fundamental movement&~inherent movement patterns 
which form the basis for specialised complex skilled 
movements. 

iii. Perceptual abili ties«~ways in which stimuli are 
responded to by the brain centres when the learner is 
making a response decision. 

iv. Physical abilities— the functional characteristics used 
when an individual makes skilled body movements. 

y* Skilled movement— the result of acquisition of a degree 
of efficiency when performing a complex movement task. 

vi. Nondiscursive communication— communication by body 
movements rather than by spoken or written language. 



USBFULNBSS OF BEHAVIOaRAL OBJECTIVES 

Educational objectives specify the planned outcomes of some 
process of learning. As such they identify the purpose of 
instruction to both teacher and student. 

The main values of clearly stated, unambiguous objectives are 
that they enable: 

the teacher to know what content to teach and to what level 
of detail 

the students to know what they are expected to be able to do 
following instruction 

. the test developer (usually the teacher) to know what type 
of questions are appropriate to assess whether or not the 
students have mastered the objectives. 

In most TAPE courses objectives can be partitioned into three 
components f namely: 

performances 

conditions 

standards. 



40 

47 



These three components are variously described by other synonyms 
in the literature as reported by Swezey (1981) in Table 1. 



Table 1 

SYBOHYMS TOR COHPOHEHTS OF AN OBJ ^ gCTIVB 



■ 

PerforiBAnce 


Conditions 


Standard 


. Activity 

Action 
. Skill 

Knowledge 
Response 
j . Task 
1 . Reaction 


Test condition 

Required equipment 
. Required facilities 

Required personnel 
. Required materials 

Environment 

Set-up 

Job condition 


Criterion 
. Cut-off 

Passing grade 
Requirement 
Go, no-go 
standards 



LIMITATIONS OF BEHAVIOaRAL OBJEBCTIVBS 

The following quotation is taken from an article by Lacey and 
Lawton (1981) in *lssues in Evaluation and Accountabil . it 
sums up the substantive criticisms of the objectives approach to 
education. Put most simply the authors are saying objectives 
take the art and heart out of teaching. 

The behavioural objectives approach rests on the 
assumption that for all teaching and learning 
situations it is possible and desirable to specify in 
advance the precise chang<>$s which will be observed (and 
measured) in pupil behaviour. The essence of a 
behavioural objective is that it must be a pre- 
specified change in pupil behaviour which is testable 
and measurable. One major objection to this is that in 
some kinds of learning situations it is neither 
possible nor desirable to predict pupil responses: in 
an English literature lesson. Cor example, the teacher 
must have ideas about the kind of response which he 
hopes for as well as those he would find disappointing, 
but these are not precise changes in bebaviour«-*each 
individual's response to a work of art is unique. 
Similarly, in a history lesson, there is often no one 
right answer to a question~-a pupil's response has to 
be rated according to several different criteria such 
as knowledge of the evidence, ability to evaluate the 
evidence and draw conclusions from it, ability to argue 
logically, etc. These may be regarded as general 
objectives, but they certainly do not qualify as 
precise behavioural objectives. Many would argue, 
however, that without such practices education would be 
greatly impoverished. (p. 230) 
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The reader's attention is drawn, in particular, to the relevance 
of the above criticisms to the problem of measuring attainment in 
the affective domain. Objectives related to such things as 
tolerance and community spirit can be likened to the history 
example. Reducing such things to observable behavioural changes 
may distort and trivialise the whole educational process. 

These limitations apply in varying degrees across TAPE. They are 
most valid in the traditional 'arts' subjects like English and 
history and also in TAFE subjects which have aesthetic components 
such as drawing and design. However,^ the limitations referred to 
apply more to teaching than testiPig. There is no need for 
testing objectives to bo constraining. The percipient English 
teacher can, for example, test in such a way that unique 
responses to essay questions can be suitably rewarded. 

Clearly many TAFE subjects are, of necessity, structured around 
sets of performance objectives which involve varying mixes of 
cognitive, affective and psychonotor skills, what TAFE teachers 
must appreciate is that objectives need to be considered as a 
part of the educational process. Their usefulness is intimately 
bound up in such things as the subject matter and teaching 
approach. 
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BXBRCISB II (1) 
CLASSIFIC/iTION OF QBjBCTIVBS 



Listed below are fifteen educational objectives. 

Identify the domain to which each belongs by writing the letter 
C, A or P alongside (cognitive = C; affective = A; psychomotor 



OBJBCTIVB DQMAIH 

Displays proficiency in the use of refrigeration, 
engineering and other hand tools 



2. Keeps work place neat and tidy 



Calculates swept volume of a car cylinder given 
stroke length and diameter of bore 



4. Writes names of all parts of an electric motor 



5. Writes comprehensive reports based on field 
observations 



6. Completes additional non-compulsory assignments 

Solders together two sheets of galvanised iron 
after being shown once 



Estimates the costs of labour and materials 
involved in cons tr uc ting a three bedroom house 
given plans and specifications 



Completes jobs in a methodical and efficient manner 



10. Judges the comparative effectiveness of alternate 
solutions to practical problems of land use 



11. Plans menus to meet pre-determined budget require- 
ments 



12. Prepares company financial statement for inclusion 
in published annual reports 



13. Uses the tools of the trade witl: perfect ease 



14. Works effectively without direct supervision 



15. Mixes paints to match faded colours of automotive 
vehicles so that they are indistinguishable to the 
unaided eye. 
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BXERCISK II (2) 
CIASSIFICATIOH OF QDBSTIONS BY OBJBCTIVK 



Read the following questions and classify each one according to 
the cognitive objective you believe it is testing. 

1. The most suitable method of joining aluminium sheeting and 
tube is by means of: 

A Fusion welding C Argon arc welding 

B Brazing D Soft soldering. 

(Answer C) 

Objective 

2. If an electric refrigerator is operated with the door open 
in a perfectly insulated sealed room, what will happen to 
the teroperacure of the room? 

A It will rise slowly C It will drop slowly 

B It will remain constant D It will drop 

rapidly. 

(Answer A) 

Objective 

3. Three wires in a lead from an electric plug are 'colour- 
coded' as shown in the diagram. Which wire comes from 
EARTH? 

A Brown C Green and Yellow 




£it':er Blue or Brown 
but not green and 
yellow. 

(Answer C) 



INTERNATIONAL THR^E CORE LEAD 
CODINQ 
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The discs in the human spine are easily damaged if they are 
stressed while the back is bent. The spine resists damage 
best when it is held straight and rigid so that it is braced 
by the back and stomach muscles. The most powerful muscles 
that can be used for lifting are in the thighs and buttocks. 

Based on the information provided, which of the diagrams 
best represents an incorrect lifting position? 

A I only c III only 

B II only D Ir II and III. 



(Answer A) 
Objective 




Explain in note form the sequence of steps you would carry 
out to revive an injured person who has a heartbeat but is 
not breathing. 

Objective 
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soup bo vino. 2 
soupbowino.1 / 




A soup bowl is floating in a kitchen sink as ^hown in the 
diagram. 

If a second bowl is placed inside the first, what happens to the 
level of water in the sink? 

A It drops slightly C It remains the same 

B It rises slightly D Ar B or C can be correct 

as it depends upon the 
size of the second bowl. 

(Answer B) 

Objective 
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SOGGBSTBD ANSWERS — BXBRCISB II (1) 



OBJECTIVE DQHAIN 

1. Displays proficiency in the use of refrigeration, 
engineering and other hand tools P 

2. Keeps work place neat and tidy 

3m Calculates swept volume of a car cylinder given 

stroke length and diameter of bore C 

4. Writes names of all parts of an electric motor c 

5. Writes comprehensive reports based on field 
observations q 

6. Completes additional non-compulsory assignments A 

7. Solders together two sheets of galvanised iron 
after being shown once p 

8. Estimates the costs of labour and materials 
involved in constructing a three bedroom house 
given plans and specifications C 

9. Completes jobs in a methodical and efficient manner a 

10. Judges the comparative effectiveness of alternate 
solutions to practical problems of land use c 

11. Plans menus to meet pre-determined budget require- 
ments Q 

12. Prepares company financial statement for inclusion 

in published annual reports q 

13. Uses the tools of the trade with perfect ease p 

14. Works effectively without direct supervision a 

15. Mixes paints to match faded colours of automotive 
vehicles so that they are indistinguishable to the 
unaided eye. ^ 

[Adapted from New South Wales Department of TAPE. Developinq a 
fpp^ //-iir^r ""^"^ """^ objectives. Sydney! undated 
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SDGGBSTBD ANSWBRS — BXBRCISE II (2) 



QOBSTION COGHITIVE OBJBCTIVB 

1 KNOWLEDGE (of methodology) 

2 COMPREHENSION (interpretation) 

3 KNOWLEDGE (of conventions) 

4 COMPREHENSION (translation) 

5 SYNTHESIS 

6 APPLICATION. 
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CHAPTER III 
TYPES OF QOBSTIONS 



WHAT THIS CHAPTER IS ABOUT 

This chapter has two purposes—to describe briefly some of the 
types of questions commonly used in classroom tests and to make 
you aware of the particular features of each type. Each question 
type is presented in a way that enables you to know under what 
circumstances it should be used and to know what factors to 
consider when the question is being prepared. 



OBJECTIVES 




After completing this chapter you should 


be able to: 


describe the types of questions most 
in assessing student learning 


commonly used 


predict which type of question is most appropriate 
for assessing a particular learning outcome 


know the guidelines for preparing 
types of questions. 


the different 



TESTS, QUESTIONS AND ITEKS 

Tests are usually made up of a number of individual questions or 
items which must be answered in a specified time. In this 
chapter the terms test question and test item will be used to 
mean the same thing, although the term itea has become the more 
commonly accepted. 



DIFFERENCES IN OBJECTIVITY OF TESTS 

The more 'objective' the test question, the less judgment that 
needs to be exercised in deciding if the answer to that question 
is correct. 

A tick (✓} in the box [t] , rather than box [F| , to indicate the 
answer to a TRUE/FALSE question is TRUE provides a situation 
which requires very little judgment on the part of the teacher. 
Tests made up of TRUE/FALSE items are therefore highly objective. 
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The difference in 'objectivity' of different test types can be 
seen in Figure 1. 



AMOUNT OF 


JUDGMENT REQUIRED IN ASSESSING THE STUDENT'S RESPONSE 


very little 
judgment 




some a great deal of 
j udgment j udgmen t 








1 


1 


II 1 


multiple- 
choice 
test 


sentence 
completion 
test 


short- structured open-ended 
answer essay essay 
test 


OBJECTIVE 




SUBJECTIVE 

TYPE OF TEST 



Figure 1 



The diagram makes one point particularly clear. Scoring each 
type of test requires different amounts of judgment by the 
marker. 

It is important to appreciate that, when an overall view of 
testing is taken, no one test type has any special advantage over 
another. As will be shown later, deciding what is best depends 
on the circumstances. 

The time required to prepare and mark a particular test is often 
an important consideration in what types of items are used. 



MAKING GOOD OSB OF AVAILABLE TIME AND RBSOORCES 

Too often teachers do not use the most appropriate test items 
because either they do not have the necessary skills to produce 
good items, or they woiry about the time it will take to prepare 
a good test. However, teachers usually do have enough time to 
make assessments, their problem is more one of knowing how to use 
the time to the best advantage. 

Coneider this example. A teacher wishes to check that his 
or her students are able to recall a series of facts, such 
as dates, names, formulae, etc. There is little doubt that 
the most efficient way of making this check is to use a set 
of objective tests items (e.g» true/false or multiple-choice 
items)» However,^ the writing of such items takes time and 
many teachers find this daunting, so they tend to use an 
essay or similar test* 
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The essay test, apart from taking a long time to mark, also 
introduces the additional problem of the teacher having to 
interpret exactly what the students meant by their answers. 
Interestingly, the additional marking time usually 
approximates to the time it would have taken to prepare an 
objective test. 



In fact when the time taken to construct a test is considered the 
situation depicted in Figure 1 is reversed to that shown in 
Figure 2. 



AMOUNT OF 
long time 


TIME REQUIRED TO CONSTRUCT 
moderate time 


DIFFERENT TESTS 

little time 








multiple- 


short-answer 


open-ended 


choice test 


test 


essay 


OBJECTIVE 




SUBJECTIVE 




TYPE OF TEST 





Figure 2. 

And what is probably more important, the objective test can be 
reused with another class at another time. When this is done the 
time saved by the teacher can be considerable. 



A lack of teacher skills in item writing can be partly reduced if 
collections of good test items are available. These collectionsr 
called ite« banks, can be used to select suitable items. 



Item banks 

An item bank is a collection of questions held in a system of 
storage (bank) from which the items can be readily retrieved. 
Each item is classified in such a way that it is possible, after 
specifying the type of item needed, to withdraw a suitable one 
from the bank. 

In its simplest form the bank can be a box of cards with 
individual items written on individual cards and suitably 
classified. More elaborate banks can involve the use of a 
computer which not only stores items but has the added advantage 
of maintaining item statistics. That is, the computer can print 
out information on individual questions, such as what percentage 
of students get a question correct each time it was used, and how 
difficult it was in comparison with other questions used in the 
same test. 




TYPBS OF T£ST QOBSTIONS 

In practice, test questions do not always fall into easily 
identifiable categories. However, to make the following 
presentation clearer, the various types have been grouped 
together as set out below. 

. Practical Assessaents — in which the student performs some 
practical task based on workshop or laboratory practice. 

. Project or Field Work Assignments — in which the student 
completes some long term activity resulting in a report, 
model, large scale construction, etc. 

Written Assessments-— >in which the student produces a written 
response (varying from writing an essay to marking a box) to 
a question based on some aspect of learning. in the jargon 
of the testing industry these are often called 'paper and 
pencil tests'. 

• Oral and Listening Assessments^in which the students' 
speaking and listening skills, as they relate to an area of 
study, are being assessed. 

• Special Purpose Assessment^^in which novel or unusual test 
forms are used. 



Practical assessment 

Practical assessments involve the student performing some 
manipulative (hands-on) task in a college workshop or laboratory 
situation. Practical tests are usually done under supervision. 

Examplea of practical tests 

Given^ three unknown solids A, B and C, pttrform 
experiments to determine the solubility of each in 
water and in alcohol 

Make a loaf of bread from the ingredients you have been 
given 

Use the tools and materials provided to build a wooden 
file box. 

Although practical tests are traditionally used to assess manual 
(psychomotor) skills, such as athletic skills or typing skills, 
their great advantage is that they can also be used to measure 
abilities from the cognitive and affective areas. A properly 
designed practical test can assess in all three domains. A full 
explanation of the difference between the cognitive, affective 
and psychomotor domains is provided in Chapter II. Put simply: 
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the cognitive domain deals with knowledge and intellectual 
abilities (i.e. thinking) 

• the affective domain deals with attitudes and values (i.e. 
feeling) 

the psychomotor domain deals with manual skills (i.e. 
doing) • 

Because practical assessments can be used to assess in all three 
domains they have much to recommend them. There are, however, 
particular features which require careful consideration: 

Mastery over time 

When students are required to perform experiments, observations, 
etc* in fixed time periods (i.e. under test conditions), this is 
usually an artificial situation. A single experimental test, a 
demonstration, etc. is not typical of real life. A cook is not 
judged on a single dishl 

With most skills, the teaching process requires that students 
make several practice (training) attempts before they master a 
skill. Some students are slower learners than others and 
therefore if students are tested after, say, four practices, some 
will show mastery of the skill while others may still not have 
mastered that skill. However these 'non-masters' may eventually 
perform better on the skill than those students who mastered it 
more quickly. 

Teachers should allow an adequate number of practice attempts 
before they finally test to determine if a student has mastered a 
particular skill. 

Assessments can also be made on practice attempts but these 
Should only be usei for informing the teacher and the student on 
progress in mastering the skill. 

Objectives and record- keeping 

iPractical assessment is best done on repeated occasions over a 
period of time and requires 

a clear statement of the skills being tested. 

maintenance of careful records showing the performance of 
individual students on the skills. 

Quality of workaanship and use of standards 

i Specification of Quality 

Where a practical exercise involves the production of a precisely 
defined result it is possible to establish acceptable margins of 
error. 
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Exarrple: Aeeeeeing vhe production of a chemical from a 
known starting amount* 

After such an exercise has been completed, the 
yield of pure chemical is measured and marks 
allotted according to some predetermined rule such 
as: 



Yield of Pure Product Number of Marks 
% of possible uielcT ' 

1(0-97 4 

'0-79 2 
60^69 I 
below 60 0 



In a similar way, a micrometer can be used to check tolerances 
achieved in an exercise in metal cutting and marks awarded 
depending on the amount of deviation from the 'ideal'. 

ii Comparison with samples 

Where a practical exercise invol^^es a product that cannot be 
measured with such things as a micrometer or a beam balance, it 
may be possible for a panel of experts to agree upon a graded set 
of examples to guide the assessor. 



Use of checklists 



There is a need to have checklists to assess some types of 
practical work, and to use these periodically. These checklists 
should consider such things as: 

i approach to the task 

ability to devise a solution, plan of the work 
• initiative 

ability to give and receive instructions 

ii handling equipment 

thoroughness 
accuracy 

cleanliness, tidiness 

iii recording observations 

. legibility 
checking 
correcting 

iv translation of data 

accuracy of calculations 
preparation of report 
graphing. 

6j 
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Note that these points involve thinking (intellectual) skills and 
attitudes as well as manual skills. Each can be assessed using a 
simple five-point scale of 5 to 1 or A to E. 

For example, in assessing the way a student handlee 
equipment when making bread, or preparing a meal the 
two extremes would be: 

A checks that equipment is clean; lays 
equipment out in convenient way; shows 
considerable skill and accuracy in making 
measure s, transferring contents; well 
organised, plans ahead; works quickly; cleans 
up thoroughly on completion* 

E fa il s to ensure equipment is clean; is 
disorganised in layout of equipment; is 
clumsy, careless and inaccurate when making 
measures, transferring contents; 
disorganised, no planning ; works noisily, 
ine ff iciently ; fa il s to c I ean up on 
completion* 

Student self assessment 

Often it is as important for students to be able to assess the 
standard of their workmanship as it is to be able to produce a 
quality product. 

When it is possible to provide a set of reliable criteria, the 
student can be required to assess his/her own product. In at 
least some cases the student assessments should be check.)d by the 
teacher. Care must be taken when using this system to ensure 
that the mark/re-mark agreement of student and teacher using the 
assessment scheme is high so the teacher is confident that the 
student can assess accurately. 



Project and field work assignaents 

Project and field work differ from practical work largely in 
terms of the environment in which they take place. They are 
often done outside the college and time allocations are usually 
generous, or should be. They also normally require a lot less 
teacher supervision. 

Examples of projects and field work 

Survey the neighbours in your street to find 
out their attitudes to smoking and alcohol* 

Measure the flow of traffic past your house 
between 7*00 p*m* and 8*00 p*m each day for a 
week and explain the reasons for any 
variations you observe* 
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Scope of activities 



A project is in effect a large-scale practical exercise from 
which time constraints have been largely removed. A very wide 
range of work is covered by the term 'project' and it can involve 
activities such as 

collecting and analysing data 

engaging in open-ended experiments in a laboratory 
. making complete objects or parts of a larger project 
identifying and trying to solve problems. 
Choosing projects and field work assignments 

It is an advantage if students have some part in selecting their 
projects (perhaps from a list drawn up by the teacher). Because 
of the considerable effort that needs to be put into successfully 
completing this type of work, the element of student choice and 
hence student interest and satisfaction are important. 

Group work 

The various activities which make up project and field work 
assignments may be performed by individuals or by groups. Just 
which alternative is used needs to be taken into account when 
assessments are made. 

Interim reports 

Most project work tends to emphasise the finished product at the 
expense of the processes that have gone into achieving that 
product. Every effort should be made to assess the process 
skills even though this can be difficult, particularly if the 
work is being done in different places. One way to assess 
process is to call for interim progress reports, to initiate 
discussions with students, and to use a simple checklist to 
summarise how an individual is progressing. 

Role of teachers 

It is possib:e to take two rather extreme views of the teachers' 
role in project work. They can either provide the students with 
no help at all or alternatively treat the task as a largely 
teacher dominated exercise. Neither of these approaches is 
particularly productive and what is wanted is a balance between 
the two alternatives where the teacher acts as an important 
resource while encouraging students to work on their own as much 
as possible. 

Resources 

Good project work rarely occurs if the environment has inadequate 

resources. Therefore tasks set or chosen must be realistic the 

teacher has an important role in deciding what is possible. 
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Assessing for understanding 



In many projects it is inevitable that the students will have 
obtained help from a variety of sources, e.g. textbooks, parents, 
other students etc. Wnat therefore becomes important in 
assessing proj ect work is finding out whether the student 
understands what he or she has done. Oral assessments are a 
useful way of finding this out. (See page 74) 

Exercise 111(3) on page 80 provides an example of project 
assessment. 



Written assessnents 

The types of written assessment it<i»BS used in TAPE range from the 
highly subjective to the highly objective — from items in which 
the students write an essay to those where they simply write a 
letter or tick a box. 

The categories of written assessment items which will be 
considered in this section are: 

Essay 
C'-ructur - 
Short An 
Multiple 
True-Fcilt. 
Matching . 

This is by no the c:^&plete set of written items used in 

assessment^ however, it dU.^s give the more commonly used ones. 
Readers inter edited in exploring other types should consult some 
of the texts on test construction listed in the references on 
page 77. 

Essay questions 

Essay questions are also known as 'extended response' and 'open 
ended' questioned. The characteristics of the essay are its 
emphasis upon freedom of expression and creativity as well as 
upon depth and scope of knowledge. Hence, essay questions are 
usually used for measuring higher order cognitive abilities; for 
example, a student's ability to focus his/her knowledge on a 
problem, to relate and integrate relevant data, and to organise a 
coherent response. 

The more creative forms of essay writing have limited 
applicability in many TAFE ar'^as but teachers should not dismiss 
this form of assessment lightly. Creative wr iting can be a 
powerful source of stimulus for other classroom activities. A 
poetry writing assignment for the plumbing apprentices could 
brighten up an otherwise dull day although such an activity 
should not be attempted by the f aint-**hearted. 

In essays with less emphasis on free expression the expectations 
are for an organised and reasoned argument on the set topic. 
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Open- ended Eeeaye 

Examyle I: 

Describe the methode used in larger factories to 
correct metal distortion resulting from flame 
cutting . 

Example II: 

Describe the design and operation of sprinkler 
systems installed to extinguish fires in office 
buildings^ Include in your answer details of the 
systems triggered by melting solder and breaking 
glass m 

Essay questions that are not well constructed typically give the 
student very little indication of what is required in answering.. 
For example, the essay question, 'Discuss the safety requirements 
of workers in the metal trades industry', could be answered by 
writing a book. Questions like this create problems for both the 
student and the teacher. Both have to make judgments of what is 
an appropriate or satisfactory answer. 

A suggested list of abilities that can be usefully assessed by 
essay questions include: 

explaining a cause and effect relationship 

describing applications of principles 

presenting relevant arguments 

formulating tenable hypotheses 

formulating valid conclusions 

stating necessary assumptions 

explaining methods and procedures 

describing the limitations of data 

producing, organising and expressing ideas 

integrating learning in different areas 

. creating original forms (e.g. describing an experiment) 

evaluating the worth of ideas 

. selecting important facts and ideas. 

(From Gronlund, 1976) 
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Guidelines for preparing essay questions 

If you choose to use an ax tended^response essay question you 
should be careful to ensure that: 

i students are not being required to simply recall 
knowledge or information they have learnt. (An 
objective type of test will be a better way of doing 
this.) 

ii students know what is expected of them. The choice of 
the lead word is particularly important: 'Describe' 
and 'Discuss' are very general words and usually allow 
for broad answers. 'Explain' is more specific, and 
'Explain the relative contributions of ... ' even 
more so. 

ii i in answering tne question the student will be able to 
show his/her abili ty to descr ibe, analyse, argue, 
synthesise and summarise. It is in assessing these 
abilities that an essay question is particularly 
appropriate. 

iv if the essay is testing language skills, then some 
guidance should be g iven of the range of responses 
required. For example, the question should clear ly 
indicate whether narrative, descriptive, argumentative, 
etc. , answers are required. 

V if aspects like spelling, punctuation, content, 
clarity, sequence, style, etc., are being mark ad, then 
this should be made clear to the student. 

vi the student should know how much time there is to 
answer. Particular care must be taken to ensure that 
the question can be satisfactorily answered in the time 
available. 

Note: 

Points iv, V and vi r.ll relate to the DIRECTIONS 
FOR ANSWERING which should be a preamble to any 
test. 

vii the item does not begin with the words 'What', 'Who', 
or 'List', as these do not usually require more than a 
short response. 

viii unless what is being measured is an attitude or the 
defence of a position, do not ask: 'In your opinion' 
or 'What do you think'. 

ix the question is 'tested' by writing an ideal answer to 
it. This allows the test constructor to check on the 
reasonableness of the question and the adequacy with 
which it is stated. 
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If possible, have someone else who should be able to 
answer the question try to answer it. Comparison of 
the two answers should show up any weaknesses in the 
question or suggest additional ways of improving it. 

Criteria for marking should also be prepared at the time the 
questions are written. This means writing down what things will 
be rewarded and what things will be penalised. Different 
weightings are usually applied to different points made in an 
answer — the ' ideal answer' referred to in ix of the above 
guidelines is a good starting point for establishing the criteria 
for marking. 

Structured essay questions 

Structured essay questions are one way of removing any doubt the 
students may have about what is required of them. A good 
structured essay presents a well-defined task that improves the 
reliability of marking while at the same time allowing the 
students sufficient freedom to organise and express their 
thought^,. The form and scope of answers are limited because 
students are told specifically the context that their answers are 
to take. 

Structured Essays 

Example J; 

With the aid of a diagram, explain the operation 
of the^ soda-acid type of fire extinguisher. Label 
the diagram* Write the equationfsj showing the 
reaction that takes place when the extinguisher is 
put into operation* 

Example II: 

Four of the methods used to correct distortion of 
metals after flame cutting are: 

hydraulic presses 

beam benders 

stretching by hammer blows 
contra^heat . 

Consider the use of these methods in typical Jobs 
and explain their usefulness in terms of: 

the availability of equipment 

metal thickness 

likely damage to materials and 
the amount of time taken. 



62 

68 



Guidelines for preparing structured essay questions 



I£ you choose to use a structured-response essay question, you 
should: 

avoid lengthy explanations of what is required; structured- 
response questions often become too wordy in an attempt to 
clarify what is wanted. 

the points made under Guidelines i to ix should also be 
considered. 

Short ansver questions 

Short answer questioned are simple direct questions which can be 
answered by a number? a word, a short phrase or, at the most, a 
sentence or two. 

Included in this group are sentence completion (or missing word) 
items and cloze items. The term 'cloze' comes from the idea of 
closure in psychology. Closure refers to the human tendency to 
fill in or complete something that appears to be incomplete. For 
example, 'By eliminating every fifth in a sentence and 

the students to write the missing word, we get 

an accurate idea how well the passage been 

understood' . 

(A7.swer: By eliminating every fifth word in a sentence 
and asking the students to write in the missing word, 
we can get an accurate idea of how well the passage hae 
been understood.) 

Completion tests can be particularly useful in measukiing the 
student's retention of specific ideas or in aeoessing 
mathematical computational skills. 

Short answer questions lie ne^^r the middle ground between the 
more objective and more subjective items. There should be few 
judgments to make about the answers to a short answer question. 
The marker's main problems should i>e confined to such things as 
deciphering the students' writing and deciding .iat to do about 
things like spelling and punctuation errors. 

Short Answer Questions 

Example I 

Natt'^ the three basic operations performed on a lathe 
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Examp le II 




The reading on the inside micrometer shown is 



Guidelines for writing short answer questions 

Things to remember when preparing Short Answer Test items are: 



ii to be particularly careful in using sentences taken 
from textbooks. These often need their context if they 
are to make sense. 

iii to be pirecise. 'Who was Bob Menzies?' is a vague 
question (and has a rivUber of answers). 

iv when some precision or degree of accuracy is required, 
as in science or maths calculations; to make it clear 
what level of precision is required (e.g. the number of 
significant figures). Similarly,' if units of 
measurement are required, say so. 

V when requiring completion of sentences; not to leave so 
many gaps that the sentences become puzzles. 



Multiple choice questions 

The most popular of the objective test items is the multiple 
choice question. 

In these questions the students must choose one answer from among 
a given set of answers. Often there is one definite answer, but 
sometimes the task is to choose the best answer, or to choose the 
one incorrect answer among the choices. Once again it is 
important that the students know precisely what is expected of 
them. 

Unfortunately, as we so often find in education, test developers 
have created their own special language to describe the 
questions. To understand this language, it is necessary to know 
the following definitions. 



i 



to ensure there is a definite correct answer. 



For example, a poor sentence completion exercise 
would be: 



Welding is a 
with 



which involves 
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stimulus Material: 



Special information on which an item is based; this may be a 
paragraph, a diagram, a photograph, etc. Stimulus material 
is not always necessary. 



The part of the item which states the problem. Stems may be 
questions, directions, or incomplete statements. 

Alternatives: 

The choices from which the student is expected to select 
his/her answer. (Sometimes referred to as Options.) 

Most multiple choice items usually offer 3 to 5 alterriati. ves 
with 4 being the most common. 

Key: 

The correct alternative 
Distracters: 

The incorrect alternatives. 



Stem: 



Multiple-- choice Question 



The scientific name of the domestic 
cat is Felis domestical 




The word ^ domestical classifies 
the cat ' s 




D 



C 



B 



A 




Alternatives 
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Guidelines for writing multiple choice questions 



The points to consider when constructing multiple-choice items 
are: 

i present a simple, well defined problem in the stem. 

Example of a poorly^-defined problem 

Of all industrial accidents in Australia the most 
serious types are those involving 

A muscle strains 

B burns 

C electrical shocks 

D cutting equipment • 

The stem of this item has not defined the problem. We 
do not know the basis upon which to decide an answer. 
It could be which is most costly to the economy/ most 
likely to cause death/ most likely to cause permanent 
incapacity to individual workers/ etc. 

ii ensure that the alternative answers all relate to the 
stem in some way. 

Example of poorly-related alternativee 

Which one of the following men invent sd the 
telephone ? 

A Bell C Pasteur' 

B Salk D Marconi • 

This question is concerned with an inventor in the 
field of communications; only Bel', and Marconi were 
inventors in that field. Salk and Pasteur were 
biologists. Thus/ two of the distracters offered do 
not relate to the stem. 

iii include as much of the item as possible in the stem. 

The stem should contain the central problem so that the 
students will have some idea as to what is expected of 
themr and some tentative answer in mind/ before they 
begin to read the alternatives. 
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Example of how to improve an item by 
tranef erring information into stem 

Contrast the poorly constructed item: 
Pine Knots 

A should be covered with shellac before 
painting 

B will stop giving off pitch if they are 
treated with raw linseed oil 

C can sometimes be sealed with turpentine 

D do not need any protective coating* 

with the improved version: 

In order to avoid paint discolouration, when painting 
pine boards containing many knots, which one of the 
following would you use to cover the knots? 

A shellac C turpentine 

B raw liveeed oil D lacquer. 

the stems of the questions should be brief and to the 
point. (A mathematics item should not become a test in 
read ing comprehens ion) • 

Illustrations are often useful in presenting the 
problem simply and clearly. 

Example of how illustration can improve a 
question 

Contrast : 

A L Lanting out on a board that intersects two 
perpendicular surfaces, but not two parallel surfaces, 
is properly called 

A a bevel C a slant 

B an angle D a charii/er. 
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with the following : 




In the drawing, the shaded part of the board is called 

A a bevel C a slant 

B an angle D a chamfer^ 

V the alternatives must be grammatically consistent with 
the stem. Too often test writers provide clues to the 
answer by not attending to this requirement. 

Example of an item with bad grammar 

Soda^ash fire extinguishers must NOT he used with 
an 

A petrol fire C wood fire 

B electric fire D plastics fire. 

The use of "an' eliminates A, C, and D as possible 
answers. 

vi make sure that the answer to any item cannot be found 
in other parts of the test. It is important to 
consider the test as a whole, because information 
supplied in one item can give away the answer to an 
item elsewhere in the same test. 

vii do not include irre ^vant or unnecessary information in 
the stem. This confuses the students as they must deal 
with the irrelevanciws as well as the subutance of the 
question and the result is a 'tr ick* question. 

viii avoid using items with double negatives. 

It is very confusing to be asked to realise that it is 
not true that Huon Pine is not grown in Tasmania. 
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avoid using items where the student must choose the 
incorrect alternative. 

Sometimes it is muc:i easier to think of three correct 
alternatives to a question than three incorrect ones. 
Indeed sometimes it seems to be impossible to think of 
three incorrect statements and hence the need to use 
this format occasionally. 

However r including a large number of such items in a 
test can introduce an element of confusion as the 
students switch from negative to positive to negative 
and so on. 

When writing negative items, always emphasise the 
negative and try to put it at the end of the stem. 

Example of using emphasis in questions asking for 
incorrect alternative 

All of the following are desirable practices when 
preparing multiple choice items, EXCEPT 

A stating the stem in a positive form 

B using diagrams to cut down on the number 
of words 

C emphasising any negative words that are 
used 

D shortening the etem by lengthening the 
alternatives • 

avoid mutually inclusive and mutually exclusive 
alterna tives as much as possible. (i.e. 'all of 
these' r 'none of A, or C\ 'A and B only' r etc 

should be used spar ing ly) . 

avoid mixed sets of alternatives which bear little 
relation to each other. These are really a set of 
true/false questions in disguise. 

Example of a multiples-choice question with an 
incorrect mix of alternatives 

Alt of the following are true EXCEPT onj. Wh%ch one? 

A aluminium expands more than copper for the 
same increase in temperature 

B a temperature of O^C is equivalent to a 
temperature of 272^K 

C heat energy is used to produce electrical 
energy in a thermocouple 

D water boils at a I owe)* temperature on high 
mountains because of the humidity. 
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This question covers a wide range of topics in 
'physics', namely expansion of metals, temperature 
conversion, energy conversion and effects of air 
pressure. They bear little relation to each other. 

(Note that each alternative would be suitable as a 
true-false item.) 

xii try to have all alternatives approximately the same 
length or vary the length of the correct answer to 
eliminate length as a clue. 

There is a tendency for the correct answer to be longer 
than the other alternatives because of the need to 
qualify statements to make them unequivocally correct. 
This, of course, provides a clue to the test-wise 
student. 

Example of item in which length of alternatives 
gives clue to answer 

The term * side^effect' of a drug refers to 

A additional benefits from the drug 

B the chain effect of drug action 

C the influence of drugs on crime 

D any action of a drug in the body other 
than the one the doctor wanted tke drug 
to have. 

To overcome this problem it is often necessary to 'pad 
out' some of the incor rec t alternatives and to 
deliberately write long incorrect alternatives in other 
questions. 

xiii vary the position of the correct answer in a random 
manner. 

The correct answer should appear in each alternative 
position (A, B, C, or D) approximately an equal number 
of times. Always check through a test jefore 
administer inq it to ensure that correct answers appear 
randomly. 

xiv list the alternatives in some logical order, if this is 
applicable. 

Numerical alx.cnatives should be in either ascending or 
deconding order. 
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Examples of how to 'ordev^ numerical al'-evnativee 



The product of 22 and S is 

A 7 C 60 

B 27 D 225 

OB 

The product of 22 ai:d 5 is 

A 225 C 27 

B 60 D 7. 

True-False Questions 

Basically, this type of item presents a question to which the 
student responds by choosing between two alternatives, of which 
only one is correct. The alternative answers are usually 'true' 
and 'false', but other forms which may be used are: yes-no; 
right-wrong; correct- incorrect. 

True-'False Questions 

Circle the correct answers: 

I Air is a mixture of gases T F 

II Hydrogen sulfide is 

insoluble in water T F 

Guidelines for writing True-False ^^uestions 

When constructing True/False items be sure that you: 

i keep the statements brief. 

ii use statements where there is no exception to them 
being true or false. Avoid partly true or partly false 
items. 

iii avoid including two or more ideas in one item. 

iv avoid including trivial items. The true/talse item 
should be used to encourage thoug h t-pro voK ing 
responses. 

v avoid using words like 'all', 'always', 'no', 'never'. 
(The test-wise students realise these are usually 
false) . 

vi avoid broad general statements. Most broad 
generalisations are false unless qualified, and the use 
of qualifiers provides clues to the answer. Qualifiers 
such as * usually' and 'sometimes' generally come with 
true statements. 
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vii avoid using statements lifted straight from textbooks. 

Rephrase or modify the wording from a textbook so that 
you are measuring understanding rather than rote 
:memory. Avoid wordings that are learned by rote, by 
presenting specific applications. 

Example of how to improve a rote^memovy true-- false 
question 

For every action there is an equal and opposite 
reaction* 

T F 

is a poor item, hut 

When a hand pushes on a wall with a certain force, 
the wall pushes back on the kund with the same force. 

T F 

is acceptable m 
Matching pairs questions 

These items usually consist of two lists in which the information 
is related. The student is told how the matching is to be done. 

Matching pairs question 

Example: 

Match all the fish dishes in List 1 with the 
appropriate sauces in List 2 by writing the number 
of the sauce in the box alongside the name of each 
fish dish. 





List 1: Dishes 




List 2: 


Sauces 


□ 


sole Colbert 


(1) 


beurre 


noisette 


□ 


sole goujon frit 


(2) 


tomato 


sauce 


□ 


plaice a I 'Orly 


(3) 


beurre 


maitre d'hotel 


□ 


grilled herring 


(4) 


tartare 


sauce 






(6) 


mustard 


sauce 



[Answers from top to bottom are (3), (1), (2)3 



The content of matching exercises can be related to such things 
as techniques or processes and their uses, parts and functions, 
rules and examples, terms and definitions; or at a more complex 
level, cause-and-ef feet relationships, and theoretical statements 
and experimental bases. 
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Guidelines for writing witching pairs questions 



When writing matching-pair itefts sure that you: 

i put the w.estion ci> j^t i in the left and the response 
(answenO column o>i ^ right, 

ii give clear and precise directions (e.g., students need 
to be awars that they must match all the items in the 
left hand column, not just one pair). 

iii keep the two lists of unequal size. This prevents 
students obtaining a correct answer by the process of 
elimination. 

iv keep the number of items to be matched between about 4 
and 10. (If there are more the exercise can become 
tedious as the lists get read over and over). 

V ensure the elements in each list can be related in some 
way. 

Example of a poor matching pairs question 

Match the terms in the left hand column with 
definitions in the right hand column by writing 
each of the numbers corresponding to the 
definitions in the appropriate boxes* 

used to finish heme 

supports garment 
shape 

narrow - of woven 
fabric 

machine stitching 
with"*' n seam 
allowance 

short fibres on 
fabric surface . 

This is a poor example; the i tems in the question 
column bear little relation to one another; 'nap' and 
'selvedge' are fabric characteristics; 'stay stitching' 
is a technique used in assembling a garment; and 
' inter fac ing ' is a type of fabric used in the 
construction of a garment. Hence, little or no 
discrimination between the items is reqclred. 
Secondly, the one incorrect option in the response list 
is just a random choice eaind does not relate to the rest 
of the list except for being within the garment 
cons t^'uct ion area. 



n stay stitching 
n selvedge 

n interfacing 



(1) 
(2) 

(3J 

(4) 



(5) 



73 

7a 



Oral and l l atening assessment 



Oral assessment 

This type of assessment requires the student to make a spoken 
response to a given item. The teacher grades or marks the 
response using a checklist in much the same way as he or she 
would in a practical assessment. 

Oral tests almost always require one-to-one teacher-student 
situations. They are demanding in terms of time and can be 
threatening, particularly for the shy or nervous student. 

Oral Questions 

Ask the student to 

speak on a given topic for 5-2(? minutes 

name a piece of equipment and explain what it is used 
for. 

When setting oral tests you should remember that: 

i before giving an oral assessment the student should 
know which abilities are to be tested* You should also 
determine in advance the standards or levels of 
performance expected; 

ii some abilities will be more important than others and 
you need to be sure about the weighting given to these 
abilities. For example, reading aloud from a book is a 
relatively simple task compared to giving a talk on a 
topic; 

iii oral tests can be particularly useful as a support to 
practical, project or field assessments. By asking 
stiir).»nts to speak about their work you can find out how 
much was their own work and whether or not they 
understood what they were doing; 

iv oral assessment takes a lot of time and is best used 
for those parts of a subject where only an oral test 
will do. For example, in language work only an oral 
test can test the students* ability to pronounce words 
correctly. 

Listening assessment 

In this type of assessment the students have to listen to some 
material and are scored on their ability to comprehend or 
understand what they hear. Because students spend a very large 
amount of their time listening (both in college and away from it) 
developing their listening skills (or aural abilities) is most 
important. The ability to take shorthand is highly dependent 
upon listening skills. 
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Listening Questions 



Students are required to answer a number of questions after 
listening to 

a recording of a broadcast on some course-^related :pi.e 
a set of directions on fire drill • 
When preparing listening tests you need to remember that: 

i preparation takes time. The use of a cassette recorder 
is recommended as a reusable collection of listening 
material can be prepared. 

ii clarity of expression is important. The task of 
listening must not be complicated by poor quality 
presentation. 

iii the listening test is not meant to test the students' 
ability to memorise what they hear. Therefore, in 
preparing the recording or reading to the students, the 
material will need to be repeated— poss ibly several 
times. 



Special purpose assessments 

Most assessment in TAPE concerns knowledge, thinking skills, and 
manual skills. 

In addition to the commonly used tests already described there 
are a number of others which could be useful. Whether teachers 
use any of these will really depend on what they want to test. 
They must decide, 'Is this the most appropriate test for what has 
to be tested'? There needs to be a good reason for using other 
testing methods. However, the following two are worth special 
consideration-~other types can be found in the literature. (See 
References, page 77) 

Open book test 

Here the students may use their books, but under examination 
conditions; that is, a time limit, no talking, etc. In some ways 
it is like a project or assignment with a more severe set of 
conditions. 

This can be a very useful technique in TAPE subjects where 
students must work with manuals (e.g. roof construction manuals, 
laws and regulations). when using manuals in a typical on-the- 
job activity, the open book test is very relevant. 

Disclosed topics test 

By announcing the actual questions to be given in a test or 
examination the teacher gives the students the opportunity to 
prepare answers and ensure a better chance of success. 
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This type of test can act cs an incentive to student learning. 
One effective way of using a disclosed topics test is to ensure 
that the announced questions give a good coverage of the whole of 
the subject, but to only use a sample of these in the final test. 
Of say, 10 announced questions, the teacher may only use 5, the 
assumption being that the ability revealed in answering the 
sample would apply to the whole set. This type of test should 
not be used for objective items; it needs to stimulate the 
students to prepare quality answers, not simply to learn by rote. 



FIRAI. NOTB 

Many types of test types have been identified in the previous 
sections. 

As a general rule, a combination of teot types is better than 
using one type alone. However, this rule should not be 
interpreted too literally. A combination of test types in a 
single assessment session can be confusing to the student. The 
point is to use different methods over a period of time rather 
than all at the one time. 

When students make mistakes because of the complexity of the test 
format the test is no longer testing the objectives it set out to 
test. 

Bven in more extended testing sessions such as end-of-term or 
end-of-year examinations, when it is necessary to use more than 
one method, it is important to keep each method separate. For 
example^ there should be a section for multiple choice questions, 
another for short answers, and so on. The question types should 
not be changing randomly from one to another throughout the test. 



If 
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EXERCISE III (1) 



£ V J HPARISON OF MOL^IPLB-CHOIC^ AND 
ESSAY TEST ITEMS 



Below is an incomplete table which compares multiple-choice and essay 
test items. 

Using the spaces provided, complete the table of comparisons showing 
the advantages/disadvantages of each type. You can check your answers 
by turning to pnqe 86. 





MULTIPLE-CHOICE 


ESSAY 


UCGIL;«/ 
OutC Jn - 

Meai . : 


ooou col measuring 
knowledge, comprehension 
and application outcomes; 
generally inadequate for 
analysis, syncnesis and 
evaluation outcomes. 




J^'- pling 
oi content 


' 


The use of a relatively 
small number of items 
results in limited 
coverage, which makes 
representative sampling of 
content difficult. 
■ - 


Preparation 
of Items 


Preparation of good items 
is difficult and time- 
consuming. 




Scoring 




Subj-ctive, difficult, and 
less reliable. 


Factors 
that 
Distort 
Student' s 
Scores 


Reading ability anil 
guessing. 




Probable 
Effect on 
Learning 




Encourages students to 
organise, analyse, 
synthesise and express 
their own ideas. 
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EXERCISE III (2^ 
CHOOSING THE MOST APPROPRIATE TEST ITEM 



Below are eight objectives from a list of objectives in a basic life 
skills course. 



Select the test types which would be the most appropriate to use in 
assessing «;«3Ch objective. 



Suggested answers a^e on page 87. 


OBJECTIVE 


MOST APPROPRIATE TEST TYPES 


Recognise 1/2 and 1/4 




Find the araa of 6 rectangle in 
square metres 


~ — . 


Read bar graphs 


^ . 

> 


Recognise words that rhyme 


i 

1 


j .: esign house that incorporates 
\ energy-saving features 




Identify the gr-.ieral mood of a 
story 




Participate in group games the- « 
call for strict observance of 
rules 




Write a simple greeting for a 
forthcoming birthday of a 
relative* 
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EXBRCISB III (3) 
PROJECT ASSBSSHBNT 



Suppose the college was asked to provide a set of brick barbecues 
on a vacant block of land alongside some retirement cottc;^€S. 

Five barbecue sites are to be built, and as t^^acner-in-charge you 
have five groups of students who are each going to build one 
barbecue. 

You have also decided to assess the work of each group by 
rewarding the following aspects of the task: 

Planning and Design 

Originality 

. Quality of workmanship 

. Awareness of links between task and life in the community. 
Indicate how yot would grade each of these aspects 
1. Planning and Design ^ 



2. Originality 



3. Quality of workmanship 



4. Awareness of links 

between task and life 
in the community 



What technique would you use to assess No. 4, the 'Awareness' 
aspect? 



(Suggested answers are on page 88} . 




BXERCISB III (4) 
ORAL ASSBSSMBHT 

Consider the oral item: 

Br ief ly explain the safety rules that apply in the 
workshop/laboratory. 

(This exercise should take the student about two minutes) 
The talk is to be judged in term of its 
INTELLIGIBILITY 
ACCURACY 
VOCABULARY 

Plan a five point checklist for each of these aspects so that you 
can grade the student on each one. Indicate clearly what it is 
that you will be listening for vhen making your judgments. 

^Suggested answers are on page 89) • 
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BXBRCISE III { 5. J 
ITEM RKVIEW 



Review the items below and attempt to relate any mistakes you 
find to the suggestions on writing multiple choice items found on 
pages 66 tc 71. 

Use the space provided to explain what, if anything, s wrong 
with the item and what could be dene to improve it. 

(Note: It is not always necej=5sary to know the correct answer 
to a question to carry out a review.) 

1« Steam is 

A given off when carbon burns 
B produced by the burning of natural gas 
C a by-product of sulfur burning 
D a by-product of magnesium burain^ 
REVIEWER'S COMMENTS 



2. When writ as a percentage, 7/20 equals 
A a number less thai. 28% 
B exactly 28% 
C a number greater than 28% 
D 35% 
REVIEWER'S COMMENTS 
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Which one of the following games is the odd one out? 

A billiards C badminton 

B baseball d soccer 

REVIEWER'S COMMENTS 



The discoverer of the Law of Gravity was 

A Milton C Ronald Reagan 

B Newton d Elizabeth Taylor 

REVIEWER'S COMMENTS 



A boy has three 5 cent coins, three 10 cent coins and one 20 
cent coin. How much money does he have? 

A more than 40 cents C more than 60 cents 

B mote than 50 csnts D more than 70 cents 

REVIElrT^JR'S COMMENTS 
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The position of Poet Laureate 
relatively little work. 

The laureateship is a 

A imposition 

B sinecu.^r^ 
REVIEWER'S COMMENTS 



offers a modest Sfklary for 

C embarrassment 
O emolument 



In a class of forty childre?*, one fifth come by bus and 
three fifths walk. How many come by bike? 

A 8 C 32 

B 16 D None of the above 

REVIEWER'S CCM55ENTS 



All Of the ioilowing are true EXCEPT one. Which one? 

A otf^idity of the human stomach is due to the 

: XT; ^rnce of hydrochloric acid. 

B Bilo i& made in the pancreas. 

C Incisor teeth are used for cutting food. 

D The pancreatic duct conveys food digesting enzymes 
to the small intestine. 

REVIEWER'S COMMENTS 
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9. Which of the following compounds is a gas? 

A water C carbon dioxide 

B ethanol d silicon dioxide 
REVIEWER'S COMMENTS 




A c 

10. In the figure ABC, if AB - BC, than ABC is an 

A right angle triangle C 
B scalene triangle D 
REVIEWER'S COMMENTS 



equilateral triangle 
isosceles triangle 



(Suggested answers are on pages 90 and 91) • 
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SOGGBSTBD ANSWBRS — EXBRCXSB III (1) [Pg 78] 





MULTIPLE-CHOICE 


ICSSAY 


Learning 
Outcomes 
Measured 


Gox? for measuring 
knowledge , comprehen; ion 
and application outco:>;^s; 
generally inadequate ! r 
analysis, synthesis 
evaluation outcom*^:^ 


Inefficient for knowledge 
outcomes; good for compre- 
hension and application 
outcomes; best for 
analysis, synthesis and 
evaluation outcomes. 


Sampling 
of content 


The use of a large number 
of items results in broad 
coverage, which makes 
representative sampling of 
content feasible. 


The use of a relatively 
small number of items 
results in limited 
coverage, which makes 
representative sampling of 
content difficult. 


Preparation 
of Items 


Preparation of good items 
is difficult and time- 
consuming. 


Prepara>.f.Qn of good items 
require time and thou^-lv, 
but easier than the 
preparation of objective 
items. 


Scoring 


Objective, simple, and more 
reliable. 


Subjective, difficult, and 
less reliable. 


Factors 
that 
Distort 
Student' s 
Scores 


Reading ability and 
guessin < • 


Writing ability and 
bluffing. 


Probable 
Effect on 
Learning 


Encourages students to 
memorise and understand the 
ideas of others. 


Encourages s^udents to 
organise, analyse, 
synthesise and express 
their own ideas. 




SUGGESTED ANSWBRS — BXERCISE III (2) I Pg 79] 



OBJECTIVE 


MOST APPROPRIATE TEST TYPES 


R<;.;cgi'vise 1/2 and 1/4 


Multiple-Choice 


Find the area of a rectangle in 
square metres 


Multiple-Choice, 
Short-Answer 


Read bar graphs 


Multiple-Choice, 
Short-Answer 


Recognise words that rhyme 


Aur<.l, 

Mulciple-choice 


Design house that incorporates 
energy-saving features 


Project 


Identify the general mood o^ a 
story 


S tt* uctured-Response, 
Shot c- Answer , 


Participate in group games what 
call for strict observance of 
rules 


Pracr.;; ■ .chec^J. -- 


Write a simple greeting for a 
focthcorclng birthday of a 
relative* 

. L 


Structured-response essay, 
short-answer . 
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SUGGBSTBD AHSIiBRS — BXBRCX [Pg 80] 



Each aspect could be graded on a 1 to 5 scale as follows 

Excellent 5 

Good 4 

Satisfactory 3 

Fair 2 

Poor 1 

Assessing 'awareness* is probably best done by oral questioning. 
This can be done both while the project is underway and when it 
is completed. 

Possible questions relevant to 'awareness' are: 

Which groups do you think are going to use this barbecue? 

. Why did you choose this particular design? 

. Where are there other barbecues in the community? 

What has been the reaction of pi^ople in the retirement 
cottages to this project? 

After questioning the students, the teacher should complete an 
"awareness checklist' using a 1 to 5 scale /:s above. 

This should be done on more than one occasion. 
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SOGGBSTBD ANSWERS — BXBRCISB III (4) [Pg 81 J 



Three possible checklists are presented below. 
INTELLIGIBILITY 

4 = Perfectly or nearly perfectly intelligible, no difficulty 
in following what student says 

3 = Very clear but one or two lapses 

2 - Generally intelligible but some words cannot be 

understood or are slurred 

1 = Difficult to follow what is being said, frequent slurring 

of words 

0 = Almost incomprehensible 
ACCURACY 

4 = Includes all the ;ules and stresses more important ones 

3 = Includes all the rules but doesn't convey level of 

Importance 

2 = Knows most but not all lules 

1 = Knows only one or two rules and then imperfectly 

0 = Has only vague idea of rules, makes up inappropriate 

rules, showe disregard for safety 

V JABULARY 

4 = Words ased are suitable, knows correct terms for various 

safety equipment, none mispronounced, no inappropriate 
jargon 

3 = Words suitable, knows most of terms for equipment, one or 

two words mispronounced or used inappropriately 

2 = Words generally suitable, does not know terms for most of 

equipment, a few mispronounced or used inappropriately 

1 - Many unsuitable words used, many mispronounced 

0 3 Vocabularly extremely limit«dd and the few technical words 
used are generally mispronounced. 
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SDGGBSTEP AKSWBRS — ^BXBRCISB III (5) [ Pg 82 ] 



1. Too much detail in alternatives that should be in stem. 
REWRITE 

Steam is given off during the burning of 

A carbon c sulfur 



B 



natural gas 



magnesium. 



Two alternatives are correct (C & D). Without knowing the 
objective that is being tested it is difficult to sure 
what to do with this item. Assuming it is simply to 
accurately perform the calculation 7/20 x 100, the REWRITE 
could be 

When written as a percentccf'^ 7/20 equals 

A 7%/20 c 20% 



B 



7% 



35%. 



The question does not give the criteria by which an answer 
can be arrived at. All could be correct. For example 
billiards is the only one played with more than one ball; 
baseball is the only one in which a player is rewarded for 
hicting a ball oat of the playing area; badminton the only 
one in which racquets are used; and soccer the only one 
where the feet and head are used to propel the ball. 

Only one of the four alternatives has anything to do with 
physics so the answer can be obtained without knowing that 
Newton actually did discover the Law of Gravity. 

The actual amount of money is 65 cents; therefore, C is 
correct, but so too are B ond A as the question stands. 

This could be rewritten to ask for the 'best' answer, or to 
give actual amounts in the alternatives. 

The answer is obtained from the grammatical construct. 

Alternatives should be rf?written as 



A an imposition 
B a sinecure 



C an embarrassment 
D an emolument. 



The inclusion of d confuses the issue. Some students might 

think the question contained a trick as the remainder did 

not necessarily have co come by bike, e.g., they might have 
come by car. 
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8. The statements are not closely related. These 4 
alternatives would be better presented as four separate 
TRUE/FALSE ques t ions • 

9. Question fails to adequately define the problem. We need to 
know the conditions under which we will make our judgment 
because the temperature and pressure determine whether a 
£!ubstance is a solid, a liquid or a gas. 

REWRITE 

Which of the following compounds is a gas at normal room 
temperature and pressure? 

A water C carbon dioxide 

B ethanol D silicon dioxide. 

10. Alternatives are not grammatically consistent with the stem. 
Diagram could be improved by making it look less like an 
equilateral triangle. 

REWRITE 

In the figure ABC, if AB=BC, then ABC is 

A a right angle C an equilateral 

tr iang le tr iangle 

B a scalene triangle D an isosceles 

triangle. 
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CaAPTBR IV 



Igg? ggBCIFICATIONS 



WHAT THIS CHAPTER IS ABOUT 

This chapter is concerned witiir the development of test 
specifications by the classroom teacher. Although not directed 
specifically at those involved in TAPE Authority-wide testing the 
principles of test specification development are very similar in 
both cases. 

The chapter looks at two aspects of classroom testing, namely: 

overall planning for the assessment of the student learning 
planning for individual tests. 



After completing this chapter, you ehould be able to: 

distinguish between planning an assessment program 
for a whole subject and planning for a single test 

define what is meant by a table of specifications 
and recognise alternative approaches exist 

explain the importance of developing a table of 
specifications for classroom assessment 

n^me the steps in developing a table of 
specifications 

develop a table of specifications using a two~ 
dimensional grid with relative weights assigned to 
every cell of the grid* 



IHTRODCrCTION 

This chapter is concerned with test specifications of relevance 
to the classroom teacher. The way a teacher goes about 
developing specifications depends on whether the sub jec t is 
examined externally or internally. 

Teachers of subjects which are externally examined need to relate 
their testing to the requirements of external examiners as set 
down in the syllabus. 

When subjects are examined internally the teachers have much more 
control over decisions about what content £\nd abilities should be 
tested. 



OBJECTIVES 
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However, there also many subjects developed by teachers within 
colleges that lack syllabus statements outlining these college- 
based subjects with sufficient precision for the person 
constructing a test. The need for precision when detailing the 
objectives of a subject is discussed in Chapter II. The need is 
further reinforced in this chapter as it will be shown that all 
tests must be related to learning outcomes that are stipulated by 
the objectives in the syllabus statement. 

A traditional difficulty of students has been to understand 
what examiners and/or teachers expect of them when it comes 
to assessment. 

Students need to know how much of what has been taught will be 
tested, and what level of performance is expected. In the past 
many students would overcome this difficulty by studying old 
tests and old examination papers to gain the required 
understand ing. However, this practice should be unnecessary 
because properly developed test specifications satisfying the 
students* need to know what it is they must be able to do after a 
period of instruction. 



PLANNING TESTS FOR A WHOLE SUBJ^TT AND A UNIT OF THAT SUBJECT 

Generally, the testing of a whole subject is done by using a 
number of sub-tests each directed at a particular unit of the 
work within the subject. In such cases it becomes important to 
ensure that an overall assessment plan exists otkierwise certain 
objectives can receive inadequate treatment. The inadequate 
treatment or neglect of some objectives stems from the basic 
problems that classroom feiachers have with test construction. 



Achieving balance in a test 

Many teachers have a tendency to overload their tests with items 
measuring knowledge of isolated facts and to neglect the more 
complex learning outcomes. 

In the general s udies area^ for example^ it is not uncommon 
to include a disproportionately large number of items which 
measure knowledge of namea^ dates, places, and the like. In 
engineering, the defining of terms and the naming of 
component parts and their uses are commonly overemphasised. 
In mathematics, computational skill is frequently the only 
learning outcome measured. 

In most syllabus statements the less complex learning outcomes 
are seen as less important than understandings, applications, 
interpretations, and various other • think ing skills'. However 
they often receive undue prominence because the teacher finds it 
easier to construct test items which require the recall of this 
type of information. 
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Without a carefully developed test plan, ease of construction all 
too frequently becomes the dominant factor in selecting items and 
constructing tests. As a consequence, the tests measure a 
limited and biased sample of student ability and neglect many of 
the learning outcomes that are considered to be more important. 

When the tests set during the course have such problems the 
cumulative effect is a poor testing prooram. 



Need for an overall test plan 

Teachers need to begin their teaching program with plans to 
assess all the objectives of the course. 

Including all instructional objectives in such a plan provides a 
clear picture of what is being measured by classroom tests. This 
clarifies the important role of testing in the total teaching 
process and at the same time should prevent an over emphasis on 
testing procedures. Each assessment procedure is seen in its 
proper perspective and its role in the total program is readily 
perceived. Constructing tests to fit the course objectives and 
the particular group of students being taught are important 
elements in this overall planning. 



TEST SPBCIFICATIONS FOR A SUBJECT 

The first step in the assessment of student learning of any 
teachinq program is the setting down of test specifications. 
This involves making important decisions concerning: 

the content is to be tested 

the objectives (cognitive, affective and psychomotor) to be 
assessed 

. the weighting is to be assigned to the various elements of 
content and objectives. 

A convenient way of arriving at such decisions is to set up a 
table of specifications in the form of a test grid. 

The following table of specifications has been developed for the 
subject 'Individual Development' in a Typing and Communication 
course in NSW. 
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Table la 

TABUS OF SPBCIFICATIOHS FOR THB SUBJECT: INDIVIDOAL DEVELOPMENT 
{From Typing and Comiunication — NSH) 



^Vs^ Object- 

Content 
Topics 




Cognitive Domain 


Affective DomaiA 


Total 


Knowledge 


COBipre- Appli- 
heneion cation 


Respon- V'.luing Organi* 
ding sing 


Emphaaie 

(percent- 
age) 


Exercise and 
Relaxation 


6 


6 12 


- - - 


24 


Diet and 

Nutrition 


6 


6 12 




24 


Recreation and 
Leisure 


6 


6 10 




r2 


Self- 
Perception 






9 4 2 


15 


Inter*Pereonal 
Relationehipe 






9 4 2 


15 


Total Bnphaeie 
(percentage) 


18 


16 34 


19 8 4 


100 












70'. 


30» 





In assigning weights to the various elements o£ content and 
objectives a common procedure is to weight each area in terms of 
the time (number of weeks) to be devoted to it. The weights 
assigned to each objective should also reflect the importance the 
teacher attaches to that objective and the emphasis to be given 
to it in his or her teaching* (The emphasis will normally be 
specified or at least indicated in the syllabus document) • Thus, 
if the syllabus stresses the comprehension and application of 
principles and generalisations, the teacher should give these 
objectives correspondingly greater weight. 

The areas of content and the obj ectivee in the table of 
epeeifioations^ should be assigned weights which reflect the 
emphasis given to them in the subject. When a lot of time 
has been given to teaching a particular topic^ then that 
topic needs to be thoroughly tested and its assessment given 
a greater weight than is given to other topics^ 

This will make it possible to construct tests that measure both 
content and objectives in a representative manner. 




The assigning of 'Vumb^rs as shovn in Table la gives the 
impression; of a higP^^y objective operation. This is basically a 
false impression ^he assignment of weightings described above 
has a considerable degree of subjectivity attached to it. 
Indeed, not only mi^^t opinions about degrees of importance of 
topics be taken int^ account, but there is also the question of 
availability of te^t Items « 

A major problem fOp test constcuctors is the fact that questions 
on some topics ace ni^« difficult to write than others. There is 
no point in using ^ t^cor question in a test simply because it is 
on a topic that i0 fl^t adequately covered. A good question must 
be found or the iri^^^uate coverage of the topic accepted as one 
of the many unfair nate 'facts of life' in the art of test 
construction. 

In practice, there J<7^«, Table la would have evolved from a more 
pragmatic table ot specifications that looked something like 
Table lb. 

Table lb 

TABLE OF SPBCIPICA^I^^S POR THB SOBJBCT: INDIVIDUAL DEVBLOPMBNT 



Content 
Topics 



Object- 
ives 



Exercise and 
Relaxation 

Diet and 
nutrition 

Recreation and 
Leisure 

Self- 
Perception 

Inter*Personal 
Relationships 



^^i^ive Dooain 



Knoi'ie^^^ Conpre- Appli-- 
lieasion cation 



^^u« 2/3 rds 



Affective Domain 



Respon- Valuing Organi- 
ding sing 



About 1/3 rd 



Total 
EiQphasis 

(percent* 
age) 



On this table the ciC^^ show the probable initial thinking of the 
test constructor. *^hls would have become more • refined* as 
actual items w^re c<7'Xsider^d and with some juggling of numbers 
the final result w^^ "^able la. 

In the classroom ^it^^tion* with limited time and resources, a 
Table of Specif ioati^^s of the type shown in Table lb is quite 
adequate for most pU^Poses. 
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CoMprefaensiveness of subject test specifications 



Subject test specifications must be comprehensive. Teachers need 
to include objectives other than those measured in written tests. 
In some subjectSr practical performance is all important. 

The table then becomes a general plan which includes the 
specifications of the overall testing program as well as 
suggestions concerning methods of assessment to be used in 
particular areas. 



Modular appr<>ach 

TAFE courses are increasingly being presented in a modular form. 
This involves breaking the syllabus up into small units which are 
taught and assessed independently. 

Consider the following part of a module from an earthmoving 
operators course. 



PERFORMANCE: Trainees will safely and efficiently use the 
ripper's on all the available bulldozers. 



MODULE: BULLDQZ ERS 



TOPIC: 



RIPPING TECHNIQUES 



CONDITIONS: 



Practical instruction . 

At least two different makes of 
bulldozer with ri^ppeys. 
Suitable practical work area. 



STANDARDS: 



To instructor' s satisfaction during 



practice . 



Satisfactory score on checklist , 



ASSESSMENT: 



By instructor evaluation of practice 
performance . 

By checklist items as follows. 
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CHECKLIST FOR RIPPING TECHNIQUES 



ITEM /or X CuMMENT 



RIPPING 

In Lowers rippers while moving. 

2 4 Uses correct number of 

rippers for ground hardness • 

3» Rips in a straight line* 

4 • Rips with the grain of the 
ground* 

5 . Adjusts ripper length to suit 
ground* 

6 . Does not turn while ripper 
is penetrating • 

?• Retracts ripper while moving* 



INSTRUCTOR SIGNATURE: DATE: 



This sub-module on ripping techniques is one of a set which makes 
up the Bulldozer Module of an Earthmoving Operators Course. By 
combining the performance objective, conditions and standards 
with the actual assessment instrument a mini test specification 
is produced. The whole module is assessed via the combination of 
the individual assessments on each sub-module. 

This approach is an alternative to a Table of Specifications as 
set out in Table la. 



TEST SPECIFICATIONS FOR A UNIT OF A SUBJECT 

Once an overall test specification has been decided upon for a 
subject it is a relatively straightforward task to develop 
specifications for testing parts of that subject. 

Test specifications for units are done in the same way as those 
for subjects except that a more detailed breakdown of both 
content and objectives is usually appropriate. Consider the 
following table for a unit on camera techniques. 
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Table 2 

TABLE OF SPECIFICATIONS FOR A 
CORRICaLON UNIT: BASIC CAMERA TECHNIQUES 



^S^^ Object- 


Cognitive Domain 


Performance 
Domain 


Total 
S&phasis 


Content ^n^^ 
Topics ^s.^ 


Knowledge 


Compre- Appli- 
hension cation 


Evaluation 


Practical 
Skills 


(percent- 
age) 


Canera Types 


4 










laage^Fomation 




3 6 


- 


- 


15 


Phot ogr aphi c 
Routine 


< 






4 


9 


Standardisation 


4 






4 


9 


Line Copy and 

Negative 

Assessment 


2 




1 




4 


Exposure Control 


4 








5 


fixed Aperture 
System 


5 


5 




8 


18 


Fixed Ratio 
System 


5 


5 




8 


18 


Development 


2 


1 


2 


4 


9 


Emulsion Charac- 
teristics 


2 


1 


2 


4 


9 


Total Efl^hasis 
(percentage) 


38 


9 16 


5 


32 


100 



Table 2 uses the approach of breaking down the content into more 
prec ise elements . 

An alternative approach is to break down the objectives into more 
specific elements as shown in Table 3 which sets out the 
specifications for a unit on weather in the subject Navigation. 

Teachers should choose the technique which is most appropriate 
for their subject. It is important to ensure that a sufficient 
level of detail exists to make it clear what will be assessed. 

The table of specifications for the subject should be made 
available to the students to help them plan their work. Whether 
or not tables of specifications for unit tests are given to the 
students depends to some degree on the subject being tested. 
Obviously for most practical tests they should be, but in some 
areas of theory this action may be inappropriate. 
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Table 3 

TABLE OF SPECIFICATIONS FOR A UNIT ON WEATHER 





OBJECTIVES 




Content 


Xnotfs 


Understands 

influence of 
each factor 
on weather 
fomation 


Interprets 

weather 

naps 


Performance S)cill in 




symbols 

and 
tens 


Specific 
facts 


use of 

measuring 

devices 


construtting 

weather 

maps 




Air pressure 


2 (6%) 


2 (4%) 


3 (€%) 


2 (4%) 


Observe 
pupils 
using 
measuring 
devices 
(Hating 
Scale 0-3 
for each) 


Evaluate 
maps con- 
structed 
by pupils 
(Rating 
Scale 0-2 
for each) 




Wind 


2 (4%) 


2 (4%) 


5 (10%) 


2 (4%) 




Teaperature 


2 (4%) 


2 (4«) 


2 (4%) 


2 (2«) 




Uuaidity and 
precipitation 


2 (4%) 


1 (2%) 


2 at) 


2 (4%) 




Clouds 


2 (4%) 


2 (4%) 


1 (2%) 










Total nuaber 
of iteas 


10 


9 


13 


8 


4 


4 


48 


Percent of 
■arks 


22 


18 


26 


14 


12 


8 


100% 



SUNNARY 

Tables of specification should be drawn up for both overall 
subject test programs and single tests. 

They are the blueprints which teachers use to build their tests- 
There are three simple steps in the construction of a table of 
specifications. 

i Down the left hand side (vertical axis) of the table list 
the subject satter divisions (content topics) to be assessed 

More detail may be added as required » 

ii Along the top (horizontal axis) of the table list the 
learaing objectives to be assessed 

These should be grouped into the major objective domains or 
categories and it is usual to use the more general learning 
objectives for these headings. More specific objectives may 
be indicated for each outcome if this detail is required. 

iii Allocate a weighting to each subject aatter and learning 
objectives category 

These weightings, wr i tten as percentages, indicate the 
relative emphasis to be given to eacb topic and objective 
and, consequently, indicate also what proportion of the 
total mark allocation is to be devoted to each category. 
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In assigning percentages it is wise to be flexible. In 
practice a weighting should be something to aim at rather 
than be treated as a mandatory figure. A little juggling of 
the percentage figures at the end is perfectly acceptable 
and is often necessary. 

Some curricula have been developed in ways which incorporate the 
test specifications into the curriculum documents as shown on 
pages 98 and 99. Provided adequate guidance is given on the 
weighting to be applied to the various test items this is an 
equally good way of planning an assessment program. 
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EXERCISE IV (1) 

PREPARING APPROPRIATE TEST SPBCIFICATIOWS FOR A SECTION OF WORK 



The set of objectives in Reading Comprehension given below relate to 
part of a bridging course in basic literacy. 

Complete the test specification grid by 3howing how you would assess the 
attainment of these objectives. (Assume the teaching of these 
objectives has taken two months to complete and the test time available 
is 3 hours. The 3 hours can be spread ever more than one day). 

Indicate against each objective the type of item or items you would use, 
the number of items that would be required^ and the percentage of marks 
you would give to tho^ particular objective. (The total percentage must 
add to 100). 



TEST SPECIFICATION GRID FOR READING COMPREHENSION TEST (BASIC LITERACY) 







ITEMS 


OF ITEM TYPE 


« OF 
TOTAL 


1 


Identify key sentence 








2 


Restate main idea 








3 


Give title to para- 
graph 








4 


Give title to poem 








5 


Identify general mood 
of story 








6 


Identify setting of 
story 








7 


Identify main charac- 
ter 








8 


Demonstr^Lte compre- 
hension of passage by 
answering 'how* and 
• why • i tems 
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mi* 


OBJECTIVE 


NO* OF 
ITBHS 


BRIEF DESCRIPTION 
OF ITEM TYPE 


Z 1 

% OF 

TOTAL 


9 


Classify ideas under 
headings 








10 


Sequence events as 
they happen in a 
story 








11 


Perceive cause-effect 








12 


Predict outcomes 








13 


Infer traits of 
character in a 
story 








14 


Distinguish between 
reality and fantasy 












TOTAL 


100 



(Suggested answers are on pages 106 and 107). 
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BXBRCISE IV (2) 



Develop a table of specifications for a unit in a subject you are 
teaching or have taught. Include both content and objectives 
dimensions and assign relative weights to every cell. 

NOTE: There is no 'Suggested Answer' for this exercise. 

In a workshop situation the individual tables would be 
the basis for group discussion. 
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SOGGBSTBD AHSWBRS — ^XBI^ISE IV (1) 



N0< 


OBJECTIVE 


NO. OF 
ITEMS 


BRIEF DBSCRIPTIOZt 
OF ITEM TYPE 


% OF 
TOTAL 


1 


Identify key sentence 


4 


Multiple-choice with items 
based on 4 short paragraphs 


4 


2 


Restate main idea 


1 


Short-answer with suitable 
paragraph for which main 
idea must be restated 


4 


3 


Give title to para- 
araoh 


1 


Short-answer requiring title 
for suitable Daraaraoh 


5 


4 


Give title to poem 


1 


choice of poem 
is important 

note the judging of this 
answer has a subjective 

o 1 orn ^ n ^ 


5 


5 


Identify general mood 


1 


Structured- response requiring 


10 




of story 




student to write on mood of 
story that has been studied 




6 


Identify setting of 


4 


Multiple-choice questions of 


4 


7 


story 

Identify main 
character 


4 


previously studied stories 
requiring identification of 
settings (4) and main 
charac ter ( 4 ) 


4 


8 


Demonstrate compre- 


20 


Multiple-choice questions 


20 




hension of passage 
by answering 'how' 
and 'why' items 




based on passages provided 
as stimulus material 




9 


Classify ideas under 


5 


Matching^pairs — give 


5 




headings 




students a number of 
headings and ask for ideas 
to be matched 




10 


Sequence events as 


1 


Short-answer — use known 


5 




they happen in a 
story 




story, provide incorrect 
sequenCx<s and ask for 
correction (alternatively, 
use multiple choice question) 




11 


Perceive cause-effect 


1 


Essay — use story that has 


10 








been studied and ask for 
reason why certain events 
occurred 
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(cont) 



SUG6BSTBD ANSWERS — BXBRCISB IV (1) 



NO. 


OBJECTIVE 


NO. OF 
ITEMS 


BRIEF DESCRIPTION 
OF ITEM TYPE 


% OF 
TOTAL 


12 


Predict outcomes 


1 


Essay — use unseen paragraph 


10 








which sets up a problem 
situation and ask for essay 
on outcome 




13 


Infer personality of 


1 


Structured-response — require 


10 




character in a 
fstory 




students to describe 
personality 




14 


Distinguish between 


4 


True/False askinq for 


4 




reality and fantasy 




reality/fantasy of previously 
studied situations 




TOTAL 


100 



NOTE: With only three hours for testing, oral testing is not 
possible. 
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CHAPTKR V 



ITEM AMALYSIS 



WHAT THIS CHAPTER IS ABOUT 

This chapter outlines the methods for analysing the results of 
classroom tests. 

Such analysis enables a check to oe made that the questions are 
effectively testing what the teachers set out to test. 



OBJECTIVES 

After completing this chapter you ehould be able to: 

know the different techniques for analysing norm-- 
referenced and criterion^ referenced testa 

explain the terms item difficulty, discrimination 
and distracter analysis 

describe a suitable method for keeping statistics 
on tcacher^made tests. 



STATISTICS, VALIDITY AND TEST TYPES 

Test statistics are important to the teacher as they provide 
objective information on how much is being learnt and how well 
subject matter is being taught. However, test statistics are 
only as good as the quality of the test items allows. The word 
that test constructors use to describe this quality is 
■ validity' . 

Although there are different types of 'validity', the most 
relevant for typical classroom, laboratory, and workshop 
situations is content validity^ —that is, the extent to which 
a test item measures performance in the subject matter of 
the course that is being taught . 

To determine whether items are valid for a particular group, the 
educational objective being tested by each item should be 
examined. Teachers must make their own judgments on the 
relevance of items for their students. 

In this chapter the issues of statistics, validity, and test type 
are related to multiple choice tests, however the principles can 
be applied to any type of test. 
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There are important differences in the way statistics are handled 
in norm-referenced and criterion-referenced tests. The issues 
are considered in the next two sections— Chapter I explained the 
difference between the test types. 



Statistics for norm-referenced nultiple-choice tests 

The validity of a test may be affected by items which are either 
too difficult or too easy for the students being tested; by items 
which do not discriminate effectively between more able and less 
able students; and by items which have unsuitable distracters. 

Three pieces of statistical analysis which all teachers can 
perform on their own tests are: 

. a determination of the difficulty of each item; 

an estimate of the disccinination of each item; and 

a distcactec analysis. 
These tnree procedures are described in the following sections. 
Difficulty 

The difficulty of a multiple-choice item is given by the 
following formula: 



% difficulty = "^"'^Q'^ of incorrect responses x 100 
total number of responses 

Each item is scored 1 for a correct response and 0 for an 
incorrect response. For summative tests such as end-of year 
examinations, items which all students answer correctly (i.e. 0% 
difficulty) or items which no student answers correctly (i.e. 
100% difficulty) contribute nothing to separating the students. 
However, for diagnostic testing purposes, such items may indeed 
be valuable. 



Test constructors in this country also use the term Item Facility 
as an alternative to Item Difficulty. The facility of an item is 
the percentage of students giving correct answers. 

Teachers need only use one of these terms and item difficulty, 
expressed as a percentage, will be the one used in this chapter. 

The higher the percentage difficulty the harder the item. 
If the difficulty^ d = 80%^ then 8 out of every 10 students 
are getting the item wrong* 
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Teachers should, however, be warned to check the way the term is 
being used when they are reading about item difficulties. 
American literature uses the term in exactly the opposite way to 
us. (See Theobald, 1974; Izard, 1977.) As if to demonstrate the 
perverseness of the test statistician? the US literature defines 
difficulty in terms of the percentage answering correctly. They 
are, in fact, using the term in the way item facility has been 
defined above. 



Discriaination index 



The discrimination index provides a measure of an item's capacity 
to differentiate more able students from less able students. On 
the basis of the total score on the test, and in relation to a 
body of knowledge or skills, students can be classified as being 
more able or less able. Typically, for an item w ith good 
discrimination, the more able students tend to choose the correct 
response whereas the less able students tend to choose one of the 
distractors. If a larger percentage of less able than more able 
students choose the cor rect (keyed) response, the item has a 
negative value for the discrimination index, and more probably 
than not, the question is unsuitable for general use. 

A simple but quite satisfactory estimate of the value of this 
index for an item may be obtained in the following way: 

Arrange the test papers in rank order of total marks and 
divide this ranked set- into approximate thirds. Disregard 
the middle group andr with the scores of equal numbers of 
students from the upper and lower groups, use the following 
formula for each item: 



Discrimination Index, DI = ■ 

N 

where R = number in upper group (1/3) who gave the correcu 
response, 

R, = number in lower group (1/'^) who gave the correct 
response, 

N » number of students in either the upper or lower 
group. 

NOTE: It is not necessary to choose exactly one third (1/3) the 
number closest to one third is quite satisfactory. 

The possible range of the discrimination index is from *1 to +1. 
It provides an estimate of the correlation between the particular 
item and the results on the test as a whole. The more positive 
the value of the discrimination index, the more closely that item 
correlates with the items of the test: as a whole. 
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ERIC 



Consider the following two examples : 



i) 32 students take a test end, for the purpose of 
estimating the discrimination index, equal groups of 
the top 10 scorers and bottom 10 scorers on the test 
are produced. For a particular item, 8 in the upper 
group and 3 in the lower group chose the correct 
response. The discrimination index for this item then 
becomes 




= -i-O.S 



ii) On the same test, but for a different item, 2 of the 
upper group and 5 of the lower group chose the correct 
response. ThB discrimination index for this item is 




^ -0.3 



It can be seen that the first item has a positive value (+0.5) 
and therefore is discriminating in the same direction as the 
whole test. However the second is discriminating negatively, 
that is, it is contributing negatively to the results of the 
test. Where this occurs, the test constructor should try to 
determine the reason. For example, the second item may be poorly 
worded or it may have two correct responses. 

For small groups such as a class size of about 20, discrimination 
index values of greater than 0.4 are desirable, and anything less 
that 0,2 is unsatisfactory. As the size of the group increases, 
the value of the discrimination index which indicates a 
satisfactory item decreases, but for no sample size is a negative 
value or a value near zero acceptable. A good rule of thumb is 
to be suspicious between 0.3 and 0.2 and reject below 0.2. 

More sophisticated techniques for the determination of 
discrimination indices are available but they take us beyond the 
needs of the classroom teacher. However, those interested in 
applying more accurate statistics should consult Exercise V(l). 
This exercise provides an alternative method for estimating the 
Discrimination Index for most classroom tests. It should be 
noted, however, that the above discussion on discrimination 
applies to situations in which a spread of scores is required, as 
in much achievement testing* This point will be taken up again 
in the section on Statistics for criterion-referenced tests. 

Distracter analysis 

In a multiple-choice item, the attractiveness of the distracters 
to the students is important. An item with a distracter which no 
one chooses or one with distracters which are too attractive, 
particularly to the more able students, should be examined for 
ambiguity. The keyed answer may not be the only correct 
response. 
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For most teaching purposes, a count of the number of times each 
response is chosen by the students provides a reasonable 
indication of how well the distracters are working. For a more 
complete analysis, particularly where the number of students 
taking the test is large enough, discrimination indices for each 
distracter can be calculated. 

AS a result of this type of analysis, items may be continually 
improved as more data become available. There is no point in 
retaining items for future tests if they invariably discriminate 
against the more able students; similarly, there is no point in 
retaining distracters that the students never choose. 

Storage of item statistics: norm-referenced tests 

Many teachers have found that their assessment procedures have 
been greatly improved by the systematic filing of item data. One 
simple way of doing this is by using a card system. A copy of 
the question is placed on one side of the card, and the relevant 
statistical information on the reverse side. An example of the 
data side of such a card is given below. 



Subject: H^fHS 
Content: Nom8€A^ " ^^P^T'fofsr 
Cognitive objective: CaHPft£HefVS/o/K/ 
Answer: 3 



Test 
date 


No. 

Tested 


Total 

Number 

Correct 


Upper/ 

Lower 

Groups 


A 


B* 


C 


D 


Omit 


Diffi- 
culty 
% 


DI 










O 




/ 


O 


o 








f 


3 


2. 




o 


lis 






iO 




7 


/ 


/ 


a 




OS 






2- 


i 


Z 


/ 
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Such a filing system facilitates the ready location of items on 
required topics and provides a continual Jheck on the efficiency 
of the items. 

Computer programs for item analysis 

Classroom teachers should also investigate the various shortcuts 
to the above procedures that are available in computer packages. 
Most computer companies offer a test analysis service that 
provides for on-going checks on individual itemr and a regular 
review of cl^ss performance. 

Teachers are strongly advised to check what is available for the 
computers to which they have access. 

Once a system has been established the time required to carry out 
item analysis is minimal and ♦.he potential rewards in terms of 
improved teaching efficiency are considerable. 



Statistics for criterion-referenced tests 

The TAFE teacher is faced with a significant problem when it 
comes to statistics for criterion-referenced tests. Some of the 
techniques available require the teacher to possess a high level 
of mathematical sophistication. Other , simpler techniques # 
require the organisation of trial test procedures that are 
demanding of resources. 

Two of the simpler proc^idur^s will be considered — pre-test:post- 
test and masters:non-masters. 

Both procedures involve the validation of items by confirming 
that a period of instruction can result in students changing from 
'novices' to 'masters'. 

Pr3- test: post-test 

This method involves the teacher developing and administering a 
test prior to beginning instruction and then administering the 
same test at the end of the instruction period. 

For any item the number of students getting the correct answer 
after instruction should be significantly greater than the number 
getting it correct before instruction. In the ideal situation 0% 
would get the correct answer before and 100% after. However, a 
much more likely result is that some would know or guess the 
answer beforehand and some would fail to get it correct after 
instruction. This failure could be through a lack of 
understanding on the part of the student, unsatisfactory 
teaching; or a conbination of both. 
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It is therefore necessary for the teacher to decide what number 
of correct answers is significantly greater. This is essentially 
an arbitrary decision on the teacher's part. The critical number 
is, of v-iouri-j, the number getting the correct answer aftet 
instruction. In the cr iterion-ref erenced situation this must be 
high. If there are no unusual circumstances to take into 
account, anything less than 80% correct should be viewed with 
concern. 

Masters : non-aasters 

This method is essentially the same as that of pre-test :post-test 
except two different groups of test takers are involved. 

The teacher needs zo be able to identify a group of individuals 
who are masters of the subject matter to be tested. Items are 
then tried out with a group of students before they have received 
instruction (non-masters) and the group of masters of the subject 
matter. 

From the point of view of the item statistics this method has the 
advantage of being relatively independent of the quality of 
instruction. (For example, a 'good' item will not reveal itself 
as such in the pre-test:post- test situation if the quality of 
instruction has been poor). However, finding a suitable group of 
'masters' presents formidable problems of organisation for most 
teachers. 

Statistical tests of significance for criterion-referenced tests 

As indicated earlier the mathematics of statistical tests for 
criterion-referenced tests is very challenging. Many of the 
approaches are still in the experimental stage. Those wishing to 
pursue this topic to a greater depth are referred to Livingston 
(1972), Berk (1980), and Swezy (1981). 

Storage of item statistics: criterion-referenced tests 

A card system similar to the one outlined on page 113 is a 
suitable way to maintain a record of the use of cr iter ion- 
referenced t3St items. 

It is also importe.nt to realise that once the validity of an item 
has been established it is not necessary to repeat the pre- 
test:post-test, or masters:non-masters validation exercise each 
time. A teacher can assume that the item will remain valid as 
long as there is no significant change in the content taught or 
the way it is being taught. 

This is another advantage of storing item statistics as the 
teacher can note an.y marked departure in item difficulties from 
one test to another. When an item difficulty changes markedly 
between administrations the teacher should look for an 
explanation of the change. 
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Validity via consensus moderation; a non-statistical alternative 



Many TAPE teachers work in relative isolation and they can be 
placed in a position of having to make assessment decisions from 
a limited range of experience. However, as has been stressed in 
earlier chapters, the task of assessment can be made more 
straightforward if the teacher is working with a clear set of 
performance objectives stating: 

the skill to be performed 

the conditions under which it must be performed 

the acceptable standard of performance to be achieved. 

It is also possible to further improve the assessment process if 
teachers can share their assessment decisions with colleagues. 
Disciissions wi th other teachers about the wor k of students 
enables a consensus to be reached on what constitutes an 
'acceptable standard of performance'. 

In some States and Territories regular meetings of teachers are 
held to arrive at an agreement on skills, conditions and 
standards through a process known as consensus moderation. 
Teachers at these meetings discuss and agree on a common 
interpretation of the performance objectives. They bring alona 
their students' work samples, the assessments of which are also 
the subject of discussion leading to consensus. 

Consensus moderation can greatly enhance the validity of the 
assessment process. When the process does not occur as part of 
the formal system, teachers should try to set up their own 
informal network. 

Other aspects of validity are taken up in the next chapter. 
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EXERCISB V(l) 
THB PHKlS) CO-KPFICIBNT AND ITEM ARALYSIS 



iNTRooocrro^ 

The C' co-efficient is one measure often used to give the 
correliution between results on one itemr and results on the test 
as a whole. On the basis of total test score, those tested may 
be allocated to an upper group or a lower g r ou^* It is 
convenient to let each group comprise one thirds or one half, of 
the total. 

The tS) co-efficient is then defined as follows: 





item 


item 




incorrect 


correct 


upper group 


B 


A 


lower group 


D 


C 


1^ ^ 


AD - 


BC 



^ (A+B) (C+D) (B+D) (A+C) 

A chart can be used to make the calculation of ^ co-efficients 
for items on a test simple matter. 

Consider the following example: 

A class of 31 students was set the following question as 
part of a larger test. 

Q. What is the maximum number of cartons measuring 
200mm x 200ram x 100mm that will fit into a box Im 
X Im X Im. 

A 1 000 C 100 

B 250 D 25 

(Answer B} 

The answers of the 31 students on the whole test were marked and 
totalled and the answer papers arranged in order from highest to 
lowest. 

The papers of the top 10 and bottom 10 students were then 
compared for each question. 
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In the case of question Q the results were as follows. 





A 


B* 


C 


D 


Number from upper group (N=10) 


1 


8 


1 


0 


Number from lower group (N=10) 


5 


2 


1 


' 1 



That is, 8 from the upper group answered correctly, but only 2 
from the lower group. 

The (5 co-efficient can be obtained by applying the formula given 
above. 



However a simpler method involves reading the value from the ABAC 
chart as shown below. 

The lines from 8 in the upper group and 2 in the lower group 
intersect at (J = 0.6. 
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BXBRCISB 

Now consider the following item in a science test taken by a 
large group of 149 students. 

Select the item out of place in the following: 
A conduction of heat 
B freezing of water 
C filtration of sand from water 
D burning of magnesium 
E refraction of a beam of light. 

(Answer D) 

After score the whole test the statistics for this particular 
item were: 





A 


B 


C 


D 


E 


N/A 


Upper group N = 50 


4 


2 


19 


13 


12 


0 


Lower group N = 50 


15 


3 


5 


2 

1 


24 

L 


1 



!• Calculate the ^ value using the ABAC chart provided below 

2* Review the item in the light of the data and suggest ways to 
improve itf if appropriate. 
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ABAC FOR THE DETERMINATION OF PHI($) COEFFICF.NT 




1. VALUE OF ^ IS 0.3 



2. REVIBHBR'S COMNBmS 



NUMBER OF CORRECT RBSPOSSBS — UPPER GROUP 



NUMBER OF CORRECT RESPONSES — liOflER GROUP 



ABAC FOR THE DETERMINATION OF PHI (^) OOSFFXCIERT 



The question has problems. 



Alternative C attracted a significant 
number o£ students from the upper group* 
It seems that they saw 'separation' of 
the two substances as sufficilent reason 
to select C. 



Since the question is about distinguishing 
between physical and chemical change the best 
solutiO)> would be to omit C altogether or to 
replace it with a less controversial example 
such as 'evaporation of alcohol'. 
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CKAPTER VI 
TEST VALIDITY AND jUSLIABILITY 



WHAT THIS CHAPTER IS ABOUT 

This chapter explains the importance of the properties o£ test 
validity and reliability as they apply to classroom tests. 



OBJECTIVES 

After completing this chapter you should he able tc: 

explain the meaning of the terms validity and 
reliability and distinfjuish between them. 

describe three different types of validity of 
importance in classroom testing. 

• explain the sources of error in teste a>:d how 
these relate to reliability. 



INTBODOCTIOH 

Chapter II explained the importance of objectives in test 
construction and how test items are written to assess (measure) 
objeccives. When a teacher writes iteias which appropriately 
assess an objective the items are said to be valid measures of 
the objective. 

Should the teacher then use these items wi):h his or her classes 
on a number of occasions over a period of time it is possible to 
obtain an astiroate of their reliability* To be classed as 
reliable, repeated testing with similar groups of students should 
produce approximately the same item difficulties. 

The properties of an item or test that make them valid and 
reliable will be explored in more detail in the following 
sections^, However/r in summary, validity is concerned with how 
well a test meets its stated purposes (objectives); reliability 
is concerned with the accuracy of the test as a measurement 
instru.nent. 

The TAPE teacher must be concerned with both these qualities of a 
test but considerations of validity are particularly important in 
a classroom situation. It is icost important that the teacher be 
confident a^out the validity of a test, however the situation 
with reliability is different. It is possible to have a highly 
reliable test which is invalid for classroom use. 
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Vov example, a mathematics teet consiating of items which 
require the addition of two numbers, each consisting nf two 
digits, juch as 

23 51 73 91 
+ + + + 
£2 2£ 19^ 

would likely prove to he highly reliable when used with 
students taking a subject such as Trade Mathematics. 

However, if the students taking the test were not taught 
addition of two digit numbers or never used this 
computational skill in their trade, then the test has little 
or no validity for them. 

Symbolically the relationship between validity and reliability 
can be illustrated with the results of a shooter firing at a 
target. 

Each time the bull's eye was the target of the shooter, yeb 
the final result was a grouping of holes off to the side as 
shown in the diagram. 




RELIABLE BUT NOT VALID 



The shooting was consistent but the shooter has not got the 
result wanted. 

Teachers must avoid such mistakes, validity is the prime concern 
in classroom testing. 



VALIDITY 

To say a particular test is 'valid' can be misleading. Tests 
cannot be ^.ermed valid by themselves — th^y have to be valid for 
something such as, *a valid test of reading comprehension' or 'a 
valid test of knowledge about legislation in th^ building 
industry'. 
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Furthermore, a single teet can be valid in a number of ways* 
Consider, for example, a maths test used as a screening test 
of applicants for places in a course. The same test could 
also be used to decide whether participant s already in a 
course n<ied some remedial assistance in nathematice. The 
validity of the test is quite different i/V the two coses . 

Test validation concerns the interpretations made of the 
results of the test. 

In this section we will consider three types of validity: 

face validity 
content validity 
predictive validity. 



Face validity 

It is possible to read the items in a test on plumbing and make a 
judgment as to whether or not they are relev?* t to a plumbing 
course. If it seems as if they are testing relevant knowledge* 
skills and attitudes then it can be said that the test has face 
validity. 

The important point to appreciate about face validity is that the 
judgments involved are superficial and m?y not stand greater 
scrutiny. Some test constructors are critical of the notion of 
face validity and suggest it could be more appropriately termed 
'faith validity'. 

Nevertheless all tests should appear to be testing what they are, 
in fact, testing because it gives the person taking the test 
greatcsr confidence. However, judgments about validity should be 
more than superficial. 



Content validity 

When the items in the test are written to assess the performance 
of students on the objectives of a subject, such a test has 
content validity. 

Content validity is of prime importance in most TAPE courses. 
For the purposes of this section it will be assumed that the 
curriculum presented to the students has 'content validity'. 

For example, we will assume the curriculum for a course such 
as the TV Technicians Certificate presents everything that 
students need when they work as TV technicians after having 
successfully completed the course. 

Our concern is with the validity of the tests used in the 
courses, the validity of the curriculum will not be considered. 
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The task of the teacher is to select content and to devise test 
situations which accurately reflect the objectives of the course. 
In devising the test situation the teacher must also clearly 
specxfy the rales that are to be applied for judging student 
performance. 

Fo7» example, a teet for TV technicians would obviously 
include the repair of parts of a television sot. However to 
be consider as having ^content validity' such a test would 
need to be accompanied by a ^lear set of instructions and 
state a time limit in ohich the task had to be completed. 

Successful completion of a TV Techni(^ians course involves 
being able to carry out vepaif^s within certain time 
Ivmits — and thin is part of the content validity the 
test* 

Content validity is largely about -^udgjuents. The judgments are 
mainly concerned with what will be externally observable as a 
result of of testing. Furthermore, the judgments are made 
without considering the persons being tested. It is the task 
that is all important. If allowances are made for individuals, 
content validity is at risk. 

In the case of our TV technician students used above, it is 
perfectly valid to fail a student who takes 30 minutea to 
repair a part that the curriculum specifies should be 
repaired in 5 minutes. 

The test grids referred to in Chapter IV are examples of the 
method by which a teacher can ensure that a test has hiqh content 
validity. Zn such cases the test must adequately sample the 
subject matter and performance objectives and reflect the ways in 
which the various elements have been taught. 

These are all judgments which the teacher is able to make. 
Indeed r the teacher is the person best placed to make judgments 
about content validity because not only does he or she knov what 
content has been taught, but also how much emphasis hafi been 
given to particular elements. 

In Chapter V, the validity related to the procedure of consensus 
moderation was content validity. 



Predictive validity 

On many occasions we are interested in how students will perform 
at sTome time in the future. Many tests are constructed with 
future performance very much in mind. A good example is the Year 
12 examination which is used to select students for places in 
tertiary institutions. Here selection is based on the assumption 
that a student who does well in the Year 12 examinations is more 
likely to be successful at tertiary studies than a student who 
does not do well. 
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The Year 12 examination is said to have predictive validity 
because it predicts success in the future. (In actual fact the 
predictive validity of the Year 1? examination is not good, 
however a certain 'folklore' has become associated with the 
examination and people perceive it as a good predicter despite 
the evidence to the contrary.} 

Any claims aboufr. predictive validity are, of course, testable and 
can be measured by seeing how well the performance on the test 
correlates with performance in the future. 

TAPE teachers are not generally fequired to give cbnsideration to 
predictive validity when framing their tests. One point to bear 
in mind, however, is that successful completion of a subject or 
course usually implies something about future performance. 

When a teacher says that a student has 'passed' the course in 
carpentry and joinery this means the teacher is making a judgment 
about how that student will perform in the trade. The teacher's 
tests whicn enabled the 'pass' decision to be made therefore have 
a predictive element in them. From time-to-time teachers should 
follow up some of their ex-students and try to reassure 
themselves that their judgments are being confirmed in the 
workplace. 



Other types of validity 

There are a number of other types of validity but they are 
unlikely to be encountered by many TAFE teachers. Those wishing 
to pursue the subject should consult the standard texts, e.g. 
Thorndike (1971). 



RELIABILITY 

The reliability of a test is concerned with the test's accuracy 
as a measurement instrument. 

All measuremenc involves error and it is the size of the error 
that determines the reliability of the measure. 

In education, the scores obtained in tests can be in error for 
many reasons. We know, for example, that assignments are likely 
to receive different scores from different markers. Furthermore, 
the same marker when asked to mark an assignment for a second 
time may well give it a different score on the second marking. 
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Error can also occur when students are asked to repeat the same 
test. Students may perform differently on the second occasion 
because 

they have learnt (or forgotten) something in the period 
between the two testings 

the test conditions have varied (e.g. noise level, weather, 
lighting) 

their state of health has changed (e.g. migraine attack, 
influenza) 

their motivation has changed. 

Teachers can also make mistakes in adding and recording marks. 

The more error that occurs in a test the more unreliable that 
test is. 

The reliability of a test can be measured in a number of ways. 
For example, in cases where it is possible to test a grjup of 
individuals on two occasions, a reliable norm-referenced test 
would generally rank the individuals in almost the same order on 
each occasion. By calculating the correlation co-efficient 
between the two groups a reliability co-efficient can be 
obtained. Such calculations are not applicable in clas?^room 
teaching situations, however teachers should be aware that longer 
tests with more items have a greater reliability than shorter 
tests. 

In the case of criterion-referenced testing, where ranking is not 
appropriate, most individuals should score the same mark or grade 
on the two occasions they take the test if it is to be considered 
reliable. 
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Thorndiker R. L. (Ed.) Educational measurement (2nd ed). 
Washi ng ton: American Council on Education. 197 i. 
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CHAPTBR VII 



RBPORTIHG riBSOLTS 



MHAT THIS CHAPTBR IS ABOOT 

The reporting of results in TAI'B is presented as an area which 
deserves much more attention than it presently receives. 

Contrast.s are drawn between what are the legitimate needs of the 
clients of the TAPE system and what they at present receive by 
way of reports. 

Suggestions are made as to how the present system of reporting 
can be improved. 



OBJECTIVES 

After completing this chapter you should be able to: 

identify the different audiences for TAFE reports 

. explain the needs of the different audiences for 
TAFE reports 

identify the necessary components of a good 
reporting system 

. explain the use of profile reporting. 



IHTRODOCTION 

A chapter on reporting results in TAFE would seem, at first 
glance, to have a relatively small amount of information with 
which to deal. 

TAFE colleges in common with Universities and Colleges of 
Advanced Education, generally provide only limited information on 
student achievement levels after the completion of subjects and 
courses. Furthermore, they provide almost no information about 
the progress of a student who is in the process of taking a 
particular subject. This level of reporting contrasts markedly 
with that available in the secondary school system where \t is 
usually possible to get term by term (or half-yearly) reports of 
a student's progress on a subject-by-subject basis. 

This chapter will describe a number of improvements that TAFE 
classroom teachers might seek to implement in their reporting 
methods. These improvements are not designed to bring TAFE more 
in line with secor^dary schools, but rather to look at new ways cf 
reporting that are appropr iate to technical and further 
education. 
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As a first step it is necessar to consider the audience the 
reports are meant to serve and wnat information a report should 
provide. 



AUDIBNCB FOR REPORTS 

Different individuals and groups have a legitimate interest in a 
student's report. The three most important are: 

the student who is the subject of the report 

the parents of the student 

employers. 

Some might argue that the student is the only one to whom a 
report should be directed. However, this fails to take account 
of the stake that others might have in the results of the 
student's education and training. For example, it is likely that 
the parents who have provided financial support will believe they 
have a right to check on their investment, or employers who have 
jobs that need filling will want help in making their selections. 
Employers can rightly argue that it is also in the students' 
interest for the most suitable applicants to be chosen for jobs. 
Therefore, while it is hard to deny that the student must be the 
first concern of the report writer, the concerns of other 
potential audiences should not be overlooked. 



The student 

A number of procedures have evolved in recent years aimed at 
protecting students from being unfairly disadvantaged by their 
reports. One way of doing this is to give students the control 
over the release of the information. This means that reports 
cannot be released to other parties, for example, a potential 
employer, without the written permission of the student. Just 
how effective such a procedure is in practice is open to 
question. Employers are likely to think twice about offering a 
position to an applicant who refuses to let them have access to 
his or her report. 

Another method is to allow students to have an input into their 
own reports. This can be done either by the student and teacher 
jointly deciding what should be reported, or by getting the 
student '^o comment in writing upon the teacher's report. 
Value o£ reports to student 

A number of studies have shown that evaluative feedback is 
positively related to increased academic performance by both high 
and low ability students (Kash and Borich, 1982). Feedback can 
vary in its value — that which occurs during a course of study is 
clearly able to affect the students' on-going performance. For 
example, a report which states there is a need for improvement 
should motivate a student to produce better work. Feedback at 
the end of a subject has less value but it can help students plan 
for the next segment of their jourse. 



134 



137 



The value of reports to students therefore centres on feedback 
and its consequent motivating influence. However, teachers 
should be aware that motivation is a delicate issue and where 
reports contain insensitive criticisms the result may well be to 
depress student motivation. 

It has already been pointed out that despite the fact that 
feedback in the form of a written report is of value to students 
there is little of it occurring in TAFE. This point is taken up 
later. 



The parents 

The parents of TAFE students are seen in a different light from 
parents of secondary school students when it comes to reports. 
In some schools the reports may actually be wr i tten to the 
parents through comments such as: 

You can be assured that the school is doing all it can to 
improve J ohn's performance in this subject. 

TAFE Students, however, are treated as mature individuals who are 
more in charge of their lives, and parents are therefore a 
secondary audience. 

Value of reports to parents 

As isidicated earlier, parents can have a legitimate interest in 
students' reports if they are providing financial support. 

At the basic level parents want to be informed about the success 
or failure of their offspring, but their needs are, in reality, 
usually greater than this. 

Parents are usually only vaguely aware of the objectives of the 
su;.^jects and course their children are taking. Therefore, if a 
report is to adequately inform parents, it needs to report in 
terms of the objectives the teachers set out to achieve. It also 
needs to be free of educational jargon so that it can be readily 
understood. 

It is therefore necessary to provide a statement of the aims of 
the course. This should not be a detailed syllabus document but 
rather a summary outline that is easy to read. Such statements 
can be printed on the back of the report form. 
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For example, consider the following statement for a Clerical Pre- 
vocational Course. 

CLERICAL PRE-'VOCATIONAL COURSE 

Communication, operative and manipulative skills 

The student can demonstrate competence in: 

Handling mail (incoming and outgoing) 

Parcelling, addressing, weighing, stamping, sorting and 
distributing mail. 

Use of a post'-book 

Use of a franking machine. 

Duplicating and copying 

Use of carbons, ink duplicators and photocopiers, and hand 
collation. 

Use of a spirit duplicator 
Use of a collating machine 
Use of ciectrical and heavy-'duty staplers , 

Filing 

Using the filing systems which are commonly available , 
Record keeping 

The maintenance of records using dockets, forms, invoices, 
books and stock record cards . 

Using records as a source of information. 

Machine operations 

Use of a typewriter for everyday operations, such as typing 
envelopes, simple memos and forms. 

Use of simple accounting machines, such as ad^-listevs, 

tills m 

(Adapted from Royal Society of Arts [UK] Vocational 
Preparation Clerical Course) 
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The eaployers 



There is a relatively high degree of compatability between the 
interests of employers and TAPE students over the matter of 
reporting results. Nevertheless employers frequently criticise 
the reports they receive for lacking important information. 
However a problem this creates for the TAPE teacher is that it 
has been shown the information employers consider to be important 
varies from job to job. The students' educational achievement is 
almost always seen as necessary information but Wilson (1982) has 
reported a considerable variation after that. Items identified 
in the Wilson survey of Victorian employers that are of 
particular interest to TAPE teachers include: 

the ability to communicate 
interest in tbz job 
the ability to work well with others 
. willingness to be flexible in job 
the desire to do well 
possessing initiative 
reliability. 

Value of reports to employers 

Employers value reports which provide valid assessments of their 
employees or potential employees. Good reports save employers 
time and money that they might otherwise need to spend on 
developing and operating their own assessment procedures. As 
with the case of parents mentioned earlierr for a report to be of 
valuer employers also need to be aware of the teaching objectives 
of the subjects reported and to have reports free of educational 
jargon. 



Other audiences 

Two other groups should be mentioned as legitimate audiences for 
reports namely: 

teachers and 

admissions officers of other tertiary institutions. 

Teachers have a dual interest. On the one hand the reports of 
student assessments provide an indication of the effectiveness of 
their own teaching and are therefore likely to have a motivating 
effect on them in much the same way as they do for their 
students. 

Teachers also value reports that have come from other teachers as 
they help them to understand the strengths and weaknesses of 
their charges. Reports are particulary valuable in the case of 
students transferring from other colleges. 
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In a similar way admissions officers of other tertiary 
institutions often require a range of information to assist them 
in deciding about selection and placement of students who are 
applying for entry with either a full or part qualification from 
a TAFB college. 



REPORTING NARKS AND GRADES 

Teachers usually give marks to pieces of work they are assessing. 
In the criterion-referenced approach common in TAFE subjects it 
is a relativly straightforward matter to give a mark of 1 when an 
item on a specific objective is done or answered correctly 
(mastered) and 0 when it is not. 

All that then needs to be decided is how many such items a 
student needs to do or answer correctly in order for the teacher, 
to be confident that the specific objective has been mastered. 
This^ problem was discussed in Chapter V where it was suggested 
that 80% mastery was appropriate in most cases — that isr 4 out of 
every 5 items done correctly. 

On the other hand, should the teacher be using a norm-referenced 
approach to assessment, marks might mean something different. A 
teacher who sets very high standards might give a mark of 50% to 
a student who he or she believes has mastered a task (e.g. 
writing an essay) and progressively higher marks for better and 
better work. The problem here is that unless we know the 
teacher's standards we do not know what the marks mean. 

This difficulty is related to the problem of interpreting reports 
referred to earlier. Not only ^ is it necessary to tell the reader 
of a report what each teachers objectives are, but it is also 
necessary to explain the meaning of any mark or grade that is 
given* 

In some subjects letter grades (e.g. A to E)^ are commonly given 
instead of marks. These enable some of the worst abuse:; of marks 
to be overcome, if a group of students are awarded an A it means 
that their work is of very high standard and meaningless within- 
group comparisons are avoided (e.g. suggesting that a person with 
99% is superior to another with 97%). 

Unfortunately it does not avoid almost equally meaningless 
between-group comparisons. However, perhaps the greatest value 
of reporting letter grades is that they can inhibit the reporting 
of composite scores. 



138 

141 



The practice of adding the marks of different Year 12 
subjects together and claiming to have something which is 
meaningful is a widespread Australian practice. Yet a 
person who buys SO strawberries and 10 apples does not think 
of the purchase as being 60 pieces of fruit or 60 
Utrapples'. But, if John gets SO out of SO for a 
Mathematics test and 10 out of SO for English some people 
have no trouble in seeing this as 60% overall for Maths and 
Bnglisn! 

In fac:, John would seem to have a flair for Maths while 
experiencing difficulties with his English, and the 60% 
completely masks these important pieces of information. 

The tertiary entrance authorities around the country seem to have 
given little thought to the consequv^nces of their continued use 
of composite scores of the Yeair 12 examinations. 

Of courje^r using a criterion-referenced approach to assessment, 
and reporting in terms of performance on objectives, will largely 
overcome the problem outlined above. 

However even when results are reported for a single subject there 
can be difficulties. For example many TAPE subjects are mixtures 
of theory and practical. Here the temptation might be to allow 
high performance on theory to compensate for low performance on 
practical or vise versa. 

Such an approach has implications similar to those already 
discussed and teachers should avoid falling into such traps. The 
task of teachers is to ensure that satisfactory completion of a 
subject really means just that and once they havc set their 
standards (be they norm- or criterion-referenced based) these are 
the standards to which they adhere. 



PROFILB RBPORTING 

The idea of student profiles has been around for a long time. 
Put simply, a student profile is a document produced by teachers 
and trainers which provides information about a student's 
knowledge, skills and attitudes relevant to a particular 
curriculum. 

Profiles have developed from attempts to provide more information 
about a student than has been available through the traditional 
examination system. 

The need for more information about stuc?ents has been a 
continuing plea of employers. Indeed, all involved in the 
transition from TAPE to work can benefit from more information, 
and profiles are one way of providing that information. 
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Although profiles are being used increasingly it is important to 
note that their effectiveness remains poorly researched. Two 
United Kingdom publications (Farther Education Unit 1982 & 1984) 
cover a widG r^nge of issues relatBd to the state of the art of 
profiles development and give details of their strengths and 
weaknesses. Among the research evidence still being sought is an 
answer to the question 'How valid are profiles'? 

While they are basically very supportive of profiles, the Further 
Education Unit publications point out that the subjective nature 
of the reporting of the attitudes and other personal qualities of 
individuals remains a problem. There are also questions about 
the level of acceptance that profile reports has among the wider 
community. 

Two examples of profiles developed in the United Kingdom follow. 
Australian TAFE teachers are encouraged to try out this approach 
in their own classrooms and workshops. 



A profile for adult instructors and supervisors 

The following profile was developed by the City and Guilds of 
London Institute and the Manpower Services Commission. 

The profile describes the course performance of men and women 
working with unemployed 16-.19 year olds who are taking part in 
the UK*s Youth Opportunities Programme (YOP). Five basic areas 
of skill are provided by the course, namely: 

Instructional Skills 
Communication Skills 
Caring/Guidance Skills 
Evaluation Skills 
Job Skills. 

The content of the course is based on a set of 22 prescribed 
objectives covering each of the five areas of skill. 

A sample^ Profile Report is provided on pages 142 to 144. 
Each^ objective has its own set of descriptors showing 
possible levels of achievement of the objectives. Minimum 
acceptable levels of achievement are shown by the HIGHEST 
level of hatching on a descriptor. For example in objective 
6, (page 142) and level 1 the descriptor: 'Fays only some 
attention to the importance of these factors' is the mir.imum 
acceptance level, whereas in objective 9, (page 143; the 
level 2 descriptor: 'Makes a fair effort to encourage 
communication' is the minimum acceptable level. 



140 



The assessment of tha achievement of the objectives is carried 
out over the full period of the course. Some of the objectives 
can be assessed during the taught part of the course. For 
example^ objective 6 can be assessed through the course 
participant prjrforming in a class exercise, case studies or role- 
playing. Objective 11 can be assessed though observation of 
discussion or a short piece of written work. Other objectives, 
for example 20 and 22, are best assessed by observing the 
Instructors/Supervisors at their place of work when they are 
working with trainees. In order to make an accurate assessment 
of objectives 18 and 19 the course tutor will consult with the 
YOP scheme manager to whom the Instructor/Supervisor is 
responsible. 

The objectives and their descriptors ate intended to be used by 
the course tutor and the course member together to enh^ince the 
dialogue, analysis and assessment relevant to the learning 
experiences of the course member. 

On the last day of the course each course member is interviewed 
and their final (summative) profile is agreed between tho coarse 
member and the course tutor (s). The tutors observation in the 
form of a supportive written comment is usually agreed at the 
same time. 

On the example shown on pages 142 to 144, the final agreed 
assessment levels are shown by a double line being drawn 
vertically in the appropriate descriptor box. 

The course tutor indicates how well the course member has 
performed within the individual descriptor by positioning the 
double lines to the left or right of the centre of the descriptor 
box as is appropriate. 

Further details of this profile approach can be obtained from 
Profiles (Farther Education Unit, Sept, 1982, pp, 12-18), 
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PRORLE- INSTRUCTIONAL SKILLS 



NAME: 



TlwtralnM 
•houMb^ibtolo: 



1 Identify and Ml0ct 
appfDpriat« inttruction 
mathods and plans for 
instnicting individuala 
and/or small groups. 

Marie prasantatatus 



2 Malceaffactivausoof 
availabia rasources for 
updating his own 
knowladga, and 
praparing instnicting/- 
iaaming matarlals. 

Mark prasantatatus 



3 Salactandusa 
appropriata rasourcas, 
Including audio-visual 
matarials, affactivaly in 
a rangs of instnicting/- 
iaaming situations. 

MaHcprasant status 



PmflkiQriid* 



klantmcation and 
salaction is rastricted In 
most casas- limirad grasp 
of methods available. 



Makes oniy partial use of 
available resources, and 
evan tftan in a limited way. 



identification and 
selection is correctHi most 
cases -some lack ilf 
vsriety. 



Makes use of easily 
available resources, 
without seeking furtfrer 
afield. 



Identification and 
aeiectkKi is correct in all 



Makes full and effective 
use of all re^rces for 
planning am preparation. 



Uses learning aids and 
resources In a very limited 
way. 



Makes use of available 
resources, but uses| 
limHed creativity. 



Uses resources artd aid to 
ttw fullest effect in the 
learning situation. 



Tutor's 
Observatlont 



PRORLE - COMMUNICATION SKILLS NAME:... 



Objectives 
nie trainee 
MfUclDr/a^wvter 
slNMiM be eMail: 


Profils Grade 


Tutor*s 


1 


2 




Observations 


4 Recognize pro^^iems 
experienced by his 
trainees in written and 
oral communications, 
and offer realistk: help. 

Mark preaent status 


Recognizes some 
problems, but has 
difficutties in offering 
remedies. 


Recognizes « 
not all prot>le 
succeed atm 
at correction. 


veral.but 
s; will 
St, attempts 


Makes full identification of 
all problems, and offers 
realistk: and helpful 
solutkxis. 














5 Demonstrate different 
ways in whk:h effective 
oommunicatk)n may 
be achieved. 

Mark present status 


Has some difficulty in 
appreciating different 
metlKxto of effective 
communication. 


Uses a fair range of 
metttods. occasionall! 
inappropriate. 




Good demonstratkxi of a 
«vkle range of 
oommunlcatkm methods. 




y//////////, 








6 Identify the purpose 
and importance of 
vocabulary, accent, 
atyteandformin 
communication. 


Paya only some attentton 
to the importance of these 
factors. 


Racognizesthe 
importance of some 
factors, but does not i 
ttiem fully. 




Stresses fully the 
importance of thttsd 
factors, and uses 
variatkxis as appropriate. 




Mark present status 










7 identify appropriate 
fonns of speech and 
wrftrfigforusa In his 
own instructing/ 
supen/isofy situations. 


Has some difficulty in 
choosing arpropriate 
fonns 'or day-to^y 
occaakKis. 


Chooses appropriate \ 
on most oocask)ns bu 
all -some difficulties i 
style. 


orm 

K>t 


Uses realistk: and 
appropriate styles on all 
occasions. 




Mark present status 
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PRORLE COMMUNICATION SKILLS - (Cont*d) 



NAME 



Objecthm 

Th« trainee 
kwlructar/a 
should be abto to: 



Profile Grado 



Tutor-* 
ObsATvatlons 



8 Use a wMe range of 
vortMrf. non-vertoal and 
visual methods of 
oommunlcation in his 
own instructing/ 
supervising situations 

Mark presiint status 



Relies on a limited lange 
of mettKxIs. with little 
undorstanding of studem 
difncutties. 



Uses a modeiate ranM of 
mettK>ds. txit lades 8( ne 
skill in modifying thef^to 
meet student need, 



yy //////// ///////A/y 



Uses a full range of 
communicstion methods 
effectively to the students' 
advantage. 



9 Demonstiate ways of 
helping his trainees to 
oonvnunicate 
effectively. 

Made present status 



Makes only a HmHed effort 
to help trainees with 
commijnication problems. 



Makes a fsir effort to 
er)courage 

oommunicatkMi. 



yyyyyyyyyxyyyyyyyyyA 



Mat « a sustained effort 
ei xMjrsge trainees to 
oonfpunicate effecthrely- 



PRORLE - CARING/GUIDANCE SKILLS NAME: 



UDjeciives 
The trainee 
instrudor/aupervtoor 
should IM able to: 


Profile Grade 


Tutor's 


1 


2 


3 


Observations 


10 Describe fully the role 
of hrS own institution ir 
the provision of 
training opportunitif>s 
for young people. 

Mark present status 


Description is limitecj 
inaccuracies, and 
expressing an incom 
view. 


with 
lete 


Description is generally 
accurate but lacks 
detailed information. 


Gives full and accurate 
description, relevant to 
the institution in question. 




yyyyyyyyyX 


^y 








11 Describe the nature 
and problems of 
emptoyment and 
unemployment, 
relative to his trainees 

Mark present status 


Presents a shallow 
amount of information 
without much helpful 
detail. 




Description is rather 
generalized, but includes 
some relevant and 
important details. 


Presents clear, accurate 
and helpful information, 
relevant to the trainees. 




vyyyyyyyyy 








12 Offer initial help and 
guidance to his 
trainees in r:«lation to 
career, education and 
personal problems. 

Mark preser' >i2tus 


Only offers occasional 
guidance, somewhat 
removed from trainees' 
problems. 


Plays P limited 
guiding/caring role. Gives 
helpful facts but has some 
problems at a personal 
level. 


Plays 
role a 
fullaf 
traine 
Help^ 


k comprehensive 
I advisor Shows a 
preciation of 
ps' problems 
Ilin all respects 




yyyyyyyyyyy 


Wyyyyyyyy 






13 Advise his trakwes 
atxMJt alternative 
sources of help and 
guklanoa. if needed 
(referral system). 


Refers most cases to 
otfier people right from the 
start, or endeavours to 
advise comprehensively 
even when not qualified to 


Makes some use of 
referral system - has 
some difficulty in selecting 
appropriate additk>nal 
help. 


Uses referre' system well. 
Shows interest by further 




Mark present status 




V/y/////// 




14 Encourage and assist 
trainees to adopt a 
positive attitude 
towards their wodc and 
their adult roles. 

Mark present status ^ 


Not very encouragir>g. 
Gh^es a little assistance if 
tfie need is pressing. 


Gives encouragement 
and ossistance on 
occasions, but is 
influenced t>y indivkJual 
trainees and their needs. 


Gives full encoura; 
and assistance to 
trainees. 


ement 


<^grt^ ^Ajt^ i^^ik 


/yyyyyyyy/ 


/yyyyyyyy 
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PROFILE - EVALUATION SKILLS 



NAME: 



Objectives 

Th« trainee 
instnictor/auparvteor 
should l>« abl« to: 


Profile Grade 


Tutor's 
Observations 


1 


2 




1 9 1 ano pan cn seuing 
performance goals for 
the trair^ees. based or 
the oOjectives of the 
organization and the 
trainees' skills. 

Mark present status 


Takes only a limited part 
due to lack of requi red sK i 
or involvement. 


Modest \ 
1 using on 
of goals 


)arttcip^.<tiQn 
y a limited range 


Fully participates in a 
! realtstic goal-setting 
exercise. 




y////////, 


/ 






16 Choose or devise 
suitable assessment 
and evaluation 
methods for (a) 
checking tralrtees' 
attair¥nent and 
progress, (b) 
discovering their 
leamir>g problems, 
and (c.) checiung own 
effectK'encss as a 
supervisoi/instructor. 

Mark present status 


Makes limited effort to 
assess and evaluate 
using simple methods. 
i>ieeas aaaitonai skiiis. 


Falls short of a full 
appreciation of the 
techniques for 
assessment. 
Tendency to rely on 
subiective methods. 




Chooses and/or devises 
appropriate methods at all 
times for ihe three areas 
stK>wn. 






y///////A 










17 Control the trainees' 
learning situation 
though effective 
feedback and 
appraisal, and by 
meeting changing 
needs. 


Loses some control by 
using methods which are 
inappropriate. 


Uses appropriate 
mettiods on some | 
occasions. \ 
Limited control of th^ 
learning situatkm. ^ 


Uses appropriate 
methods at alltimes. In full 
control of the learning 
situation. 


.1 .1 1 


V//////y 


//////}// 




18 Construdand 

pfMant ocal and written 
fpoftaontrain— a. 

Mttic praatnt jtalua 


Produces paportt wtilc 
are irvoonckarivv and 
bttdly organiied. 




Constructs and presents 
reports whksh ere 
Inoomplelit lacfcfirtal 
polish and have only 
_»MiiUedjaliie^ 


Constructs and presents 
dearly understandable 
and worthwhile reports. 










19 Opemasatisfaclorily 
Inaderical/ 
•dfniiiiatifltlve 
aytlani. 

Mule pfVi0nt status 


LackofllaiaonwHhIhe 
syatam - grudging use of 
support seivloes. 


System use 
■^aoinetimi 
supply with 

1 ,1 a II il 1 1 

mTormaoor 


»d haphazardly 


Support systems used 
oorrectVatatltimes. 




Y///////y 


II 




20 Demonstrate 
approphate skill, 
knowledge, 
competence and 
awareness of safe 
practice in an area of 
work. 


Does not demonstrate 
appropriate knowledge 
and competence in an 
area of work. 


Has limited knowledge 
and competerKe. Needs 
some updaMng. 


Demon 
approp 
and cor 

1 


strates 

iate knowledge 
ipetence m an 
kork 


fl -^Kit cUss 


Mark present status ^ 


V///////, 


'///////. 




21 State the extent of his 
own responsibilities in 
ensuring satisfactory 
and safe conditions of 
work for his trainees. 


Statement is incomplete 
because knowledge is 
poor with inaccuracies. 


Statement is generally 
accurate, but lacks 
detailed knowledge. 


Makes f 
stateme 
instituti( 


ull and accurate 
It relating to his 

1. 


fart^t o>a»tt o.w«#4^ 


Mark present status ^ 


V///////. 








22 Plan and organize 
within his area of work, 
to enable trainees to 
be most effectively 
occupied. 


Relies on very limited 
plannirig ar>d organizing of 
work. 


Plans and organizes work 
but sometimes fails to use 
them. 


Plans and organizes work 
to fullest effect 

1 




Mark present status / 




V/////// 
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A profile for a pre--vocational course 



The second example of a profile^ one used with pre-voca tional 
training schemes, was also developed by the City and Guilds of 
London Institute. 

This profile has four components as shown in the diagram. 



CONFIDENTIAL PUBUC 




Pormatlv* Summattv* 



PIflura 2 The CGLl Profiting System 



The first three components make up the formative system, the 
purpose of which is to support the student's learning by ensuring 
that their needs are identified and that the learning program is 
satisfying those needs. These components are confidential to the 
student and to the tutor. The fourth component, the final 
(summative) profile^ report, serves the entirely different purpose 
of communicating the student's attainments to a third party, for 
example, an employer. It is, therefore, a public document. 

Log book 

The Log Book which is kept by the student, provides a record of 
what she/he has learnt and experienced and forms the main input 
to reviews. 

An example of the two pages of a Log Book entry for a particular 
week follow - the entry relates to a student undertaking a pre- 
vocational nursing program. 



145 

148 

o 

ERIC 



WEEKLY RECORD OF TRAINING 



WEEK COMMENCING 



Figure 3 



WEEK NO. 



1, What work did you do and where? 

W \v«^jM^. .Ny>»^ ^^ec>; Tvux>^o^ .fioSS^fiX \j05^ )pr 

LW^kVy>o/>.^^6^J^^ 

Vle2ki^to^flM^ . Vua:^ 

CAWS. VoWNt ^Xt^ 

5t>1Smm^ , Vq Ajev>)tv* r , , v^a^^ .Su»ml SbVxv ^ 
C^jJix .<ev^\».^ ^?>H^ 



2. What did you do that was new? 



3. What Off the Job Learning/Further Education did you undertake? 

.&:S>f^..A!^M>NwsA<>.-. 

.ScrwA.. .^)j^y>)0 . -OxX /. 



A was there anyttiing you especially enjoyed during the week and irhy? 



5. Was there anything you did not enjoy and. il not. why not? 



6. Supervisors 



^1^44^ ^ f tty^fv^l 



...<^...C^€^...(/C^^ 



Trainee's Signature ' . .^'^ . Date !^ ' 

Supervlaor-sSiQrwture .^^^^^f^^ Date J. ^ r.!^ r.'^.^. 

CGL/00435, S25, «<J 1983. City and Guilds of London institute 
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Reviews 



Reviews enable student and tutor to discuss progress? and to make 
plans for future work. They are also intended to be useful for 
identifying and resolving problems and for building good working 
relationships. The review, therefore, is at the heart of the 
formative profiling process. A jointly agreed record of each 
reviewing session is maintained withi;' the log book. 

An example of a review follows. 

FOUR WEEKLY REVIEW OF PROGRESS ^ 

1 . What are the main tasks or work areas covered in xtie last four weeks? 
Care Work - bathing 
Kitchen and domestic 



Laundry 



2. What has the trainee done particularly well? 

. . *f ^. caring. Has a pleasant reassuring nature. 

Has worked well with clients on hospital visits. 



3. After discussion, we have agreed progress has been made in: 

^ .y . ^ . s t a f f Has deye 1 oped in s jB 1 f conf i den?e 



but attention needs to be paid to: 

^^^^^^^ : . °»re self assured. 

Contributing a* -^taff meetings. 



4 . The next month will be spent : 

, Working on project .*'Cpnfus^<J P.lder.ly 
Mo re i nvo 1 yeme n t u i t h bak i n^ etc. in Kitchen. 



Tramee^s Signature . . ^/'''^. Date ..^"^r.^ .^^ ^2^. . 

Supervisor/T utor Signature . . . y^. .^^:"^^^^^. Date. . . . 

CGL/00436. S25. 1 1983 City and GurWs of London Institute 
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Progress profile 



A Progress Profile is completed jointly by the tutor and the 
student every six to ten weeks. It records specific applications 
of abilities^ the examples being drawn from the student's work 
and experience^ thereby setting a context for the 'Progress in 
Abilities' statements on the grid. The 'examples' stage is 
always completed first as it provides the basis on which to judcje 
the level of ability which has been achieved. This level is 
recorded by shading-in the bar up to and including the 
appropriate level on the grid. 

Paqe 150 shows an example of a completed Progress Profile. 
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CHy&Qulldi of London Institute 



BEST copy AVAIUBLE 



FIgura 

Progress Profit 



ABIUnES 



EXAMPLES OF ASIUTIES 
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Final profile report 



The Final Profile issued when a student leaves the scheme is 

comprised of four pages. The front page provides details of both 
student and scheme; the inside pages summarise the best examples 
of ability at the highest levels attained on the grid; and the 
back page describes how the profile system works and provides 
additional information such as other qualifications achieved. 

The following 4 pages are an example of the Final Profile 
developed by the City and Guilds of London Institute. 
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Figure 5 




CITY AND GUILDS OF 
LONDON INSTITUTE 



Incorporated by Royal Charter 



Profile Report 



Founded in 1878 



NAME: 



GILLIAN ANDERSON 



AGE: 



17 



SCHEME: YOUTH TRAINING SCHEME 

COMMUNITY SERVICE AGENCY 
69 HIGH STREET 
AMBLESIDE 

FROM: AUGUST 1983 



TO: 



MAY 1984 



WORK EXPERIENCE: 

COMMUNITY AND HEALTH SERVICE WORK 

AMBLESIDE NURSING HOME 
LAKEVIEW OLD PEOPLE'S HOME 

DOMESTIC DUTIES - CLEANING : iJUJNDRY : KITCHEN AND DINING ROOM 
PERSONAL CARE OF RESIDENTS 

OCCUPATIONAL THERAPY UNIT - HELPING RESIDENTS WITH KNITTING, 
SOFT TOY MAKING AND OTHER CRAFTS 

OFF THE JOB LEARNING: 

ENNERDALE COLLEGE OF FURTHER EDUCATION 

INTRODUCTORY CARING COURSE - 

(i) WORKING IN THE COMMUNITY 

( i i ) NUMERACY/ LANCUACE 

(iii) NUTRITION AND HEALTH EDUCATION 

(iv) COMMUNITY SERVICES 



This is an end-of-scheme profile, prepared on the basis of regular consultations between 
scheme staff and the trainee. Profiles are completed in conformity with CGLI requirements, and 
this process (but not individual attainment levels) is monitored by CGLI. 
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SPECIFIC APPLICATIONS OF ABILITIES 



Figure 5 continued 



COMMUNICATION 



Listened sytopathetically to patients worries and discussed their problems with 
theni* 

Wrote letters for short sighted patients. 

Maintained laundry stock records and checked written orders. 

Followed written instructions/patterns for knitting and soft toy making. 

Prepared an illustrated booklet on 'Caring for the Elderly' after researching 
material from a variety of sources. 

SOCIAL ABiUTIES 



Worked well with other trainees and staff at Nursing Home - discussed patients 
progress. 

Bathed, fed» dressed and shaved elderly patients without supervision. 
Allocated and supervised work in occupational therapy unit. 

Quickly got to know clients at each placement - reacted sensitively to the needs 
of confused elderly patients. 



PRACTICAL & f^UMERICAL ABILITIES 



Used kitchen, laundry and cleaning equipment; also variety of craft tools. 
Used clinical thermometer; took blood pressure. 

handled wheelchairs correctly with regard to safety and comfort of occupant, 
looked meals without supervision, weighing and measuring ingredients accurately. 
Estimated and cos ted materials for craft projects. 



»ECISION-MAKING ABILITIES 



^lanned recreational programmes to suit individual patients. 

lelped organise patients outing - planning bus routes and researching required 
information on facilities. 

loped calmly with patients having epileptic fits. 
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ATTAINMENTS IN BASIC ABILITIES FIgur* 5 continued 

(Highest level from Progress tracking grid - see overleaf) . 
COMMUNICATON Bar indicates highest 

TALKiNGANO Can comaunicate effectively with a range of ^^vel attained 

USTcNiNG people in a variety of situations 

READING AND Can use instruction Banuals and can write reports 

^'^^ describing work done 

guidance, aake use of basic graphs, 
ANDDiAGRAMS ^^arts, t-bles, dr«;dngs, etc. 

SOCIAL ABILITIES 

WORKING IN Can be an active and decisive 

A GROUP menber of a group 

»»»»*» < i »sssasssassas »»»»»»»»»»»»***»*»* 

WORKINGWTTH Can asaJM responsibility for delegated tasks and 

THOSE IN AUTHORITY take initiatives 

CUENT^^^'^'^ Can anticipate and respond to tlient's needs 

PRACTICAL & NUMERICAL ABILITIES 

EQUIPMENT select and use suitable equipment and materials 

for the job, without help 

movement'' '° ^^"^ and/or complex work 

MEASURING Can measure out specified quantities of material 

by length, weight, etc. 

CALCULATING can use x and ♦ to solve whole number problems 

********* 

DECISION-MAKING ABILITIES 

PLANNING Can modify/extend given plans/ routines to meet 

changed circumstances 
*************************^*^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

ilpoi^i^TiON 9^" show initiative in s-jcking and gathering 

mtonnation from a wi ic variety of sources 
**********************^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

Can cope with unexpected or unusual situations 
**********************^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 

Sisuf?r ^ performance and identify possible improvements 

***********************^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
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HOW PROHLES ARE COMPIL ED: * ^^"^'"ued 

During the scheme the trainee has recorded his/her work and experience in a Log Book. From 
this record, specific examples of abilities in the four main areas (and the fourteen sub- 
categories) were identified and recorded at regular intervals against the PROGRESS 
TRACKING GRID shown below. 



Simple tasks ^ Comoiex tasks 

under supervision ^ ► without supervision 



« 

u 

M 

I 






Can^^a^^^ 
Can <«a< ar4 M««aa«at 


Caw tg^m aWfi*! i wan 

CanlMV '^^tnow^'N 

*a»aia* nairHct>M, 
ana t^mttmrnnm 


Can caMmyncaia a«ac«wa^ 
•«« a lan^ at aaaata « a 

Can tiaa wail Mciwiii^a»<wi 
an« can «»«• f«patn 


Can piatam a «af<at ana 
aWK*i»a a'fianan) Can 

•rawn data ana vaa a iQ 
>»aaMca anw a'*ia"«»» 


* 


M«0DMGM«M5 




Canai«iH^li«n«an« 

Can v«t *hr« <nan*an 
a> rai« ta acft« ^ c^man 
aMt 


Can if nfM aiwca Mate 
«naa> aacfam cttan* 
laaw oaMfi aM 

CanwwaimKw tnpam^n 
and r«M» ■( aiin acMm 


Can >aa'>iat ■«< waa >a« 
7>aM ctiana iaM«an« 
a.?— <^,wi«>a 

Can ta an Bcmt a tacum 
"■aw^ai aiaroi» 


CancanaaucittaMaic an« 
afumanK 

Caniaaaafai* 




i 

« 




iMft* ^ c any »aM 
Ml a (twM t 


cnaiaiaM anMca^y dowi 
Can cany Mi cttawii 


Can wa* a aann « 
Can cany ai^ Cham , 


Canca««yawiataMnatiata« 
fii^tna 

Can *acny<iM c*a»« i naaat 


Cana«Mw<»»ani>ti| *oi 
#v^v9a*aa taaat ana i§aa 

Can aw vata antf »»apon««r 












•i 
m 

4 

n 

1 

« 

U 
I 


IOUM«MT 


Can uaa t^ant Miatf la 


»a»^<w »a»»»a» »iap laitt 


Can ana w«a MiM*^ 
■an annwiantfwuinmoi 


Can «at iW ana ni«M<Uir> 


cenwnan latM <n a«H«n«an| 




CONtlKXO' 

WOVtMCtiT 


CanM cany an« Mt aaii^t 
a«fKn 


Can wM avaiyaaf na** 




Can Ma (oaw w ao Via 
ana>Bira»»>anmo>fc 


Can aa<«ann laaki taowMai* a 
*»i^aayM tcaw»*c< 




WCAftUMMG 


Can M<i aapcn Ma w«« 


laa* wnw^ team M 


Can maaitn awt loaca^a 


Can n^M a>ac>««n 








CancaiM 


Can a«g ana MM act «mm 


CanwMian* iomm 


Can MM ana MOt* ki 
aafxan awa »n>a^ hacKn* 


Can wi^^i, anaaw^ 
•acifKatt ana t^ipta facMnt 




• 

« 

a 

i 




CaM M«yaNe« at 


Mat MMonadaMn can 
««M««y «!• «a««nca at Mi»* 
#ta»ai*wi>ai* 


lackiMf aia« 


Can mea^fitana 9f*an 
c**amaa c'^i^nfancat 


Canc*aaiana* 
a«a"t"Bwnnat tfom Kiaiett 




SflMMG 


Can««a«imwn« 




Can MM iiawQi-a Mwnatai 




Can (Mow m^tmnm « aMtmg 
•>aM a M<«a «ana«y ot aoHfcat 




COM.C 


Can Mpa MM a*aff«air 


Canraai i Wi ■ i nai 
•■^*pna ftaaaf Na^ii 


Can cap« wm enanfti m 
•■Mia>««wWM 


Cantaaa^^Mwaipacnaai 


Can atia> aaMMiva ana 
aMactwa •«« 10 a»a 




s 


Mf SmTS 




IM^awaa 


•an«««i9^, ■ a>BMma» 


C an ai«atl own a»<«oim>wc> 

ana^wityaamm 


Can MMBi aia f>«we i 
pa«a»inanca aw<M»io 
Mp»OMa 





ADDITIONAL INFORMATION 

(NOT covered by CGLI Profile Scheme) 



Is well motivated and willing to learn and is a conscientious and careful worker. 
Handled cash for making small purchases for patients. 

CSE - English (2); Biology (3); Modern Studies (4) 

CGLI - Numeracy Level I 

CGLI - Caring Skills Certificate 

Voluntary work - fundraising for charitable bodies CGE. 00467. eT2 

Comnittee member of Youth Club« 



EKLC 
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Further Education Unit. Profiles . September 1982. 

Further Education Unit. Profiles in Action . October 1984. 

Kash. M. M.# & Borich, G. D. Teachers, in Walberg, H. J. (Ed.) 
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EXERCISE VII (1) 



With the help of the examples provided in this Chapter develop a 
Profile Report Form which is relevant to a subject that you 
teach . 

(There is no suggested answer for this exercise — the form should 
be the subject of group discussion during a staff development 
exercise. ) 
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INDBX 



achievement tests, 18 

affective domain, 25-6, 39, 54-5, 96-7 
affective skills, 42 
aim, 24 

alternatives , 65 
analysis items, 27, 34-5 
application items, 27, 30-3 
assessment, 2 

continuous, 9 

in TAPE, 9 

objective, 2 

subjective, 2 
assignments, 

field work, 57 

project, 57 
aural assessment, 74 



behavioural objectives, 40 
limitations of, 41 
in TAPE, 42 
usefulness of, 40 
see objectives 



classification of qr L . .,-7 
class tests, 9 
cloze items, 63 

cognitive domain, 25^6, 54-5. 96-7 
cognitive skills, 42 
completion tests, 63 
composite scores, 138-9 
comprehension items, 27-9 

extrapolation, 27 

interpretation, 27, 29 

translation, 27, 29 
conditions of assessment, 98 
consensus moderation, 116 
content validity, 127 
continuous assessment, 10 
criterion-ref6«nced assessment, 10-12 

future of, 14 

grades. 13 

level of specificity, 15 
minimal 'acceptable performance, 15 
reliability, 13 
reporting results of, 139 
scores, 13 
statistics, 114-5 
validity, 13 
criterion-referenced tests, 11, 19 
future of, 14 
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diagnostic test, 18 
difficulty (of test items), 110 
disclosed topic tes^., 75 
discrimination index. 111 
distracter analysis, 112 
distracters, 65 



educational objectives see objectives 
error in reliability, 130 
essays, 59, 78 

guidelines for preparing, 61-2 

in TAFE, 59 

open-ended , 60-1 
evaluation, 7 

formative^ Z 

qualitative, 8 

quantitative, 7 

research and, 8 

summative, 8 
evaluation items, 27, 37-8 
examinations, 9 

end of year, 9 

Year 12, 139 



face validity, 127 

facility (of test items), 110 

formative evaluation, 8 



grades, 13 
goal, 24 

item analysis, 109 

computer programs for, 114 
phi co-efficient, 118-121 

item banks, 53 

item difficulty, 110 

item facility, 110 

item review, 82-85 



key, 65 

knowledge items, 27-8 



learning objectives, 101 
listening assessment, 74-5 
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mastery, 55 

matching pairs questions, 72 

guidelines for writing, 73 

measurement, 3 

modular approach, 98 

multiple choice questions, 64-71, 78 

guidelines for writing, 66-71 



norm-referenced assessment, 10-12 

test items, 12 

grades, 13 

scores, 13 

reliability, 13 

validity, 13 
norm-referenced tests, 11, 19 

future of, 14 

statistics, 110 

Bee item analysis 



objectives, 15, 101 

classification of, 25, 44-7 

educational, 23 

general, 24 

learning, 101 

levels of specificity, 24 

performance, 24-5 

record keeping, 55 

Bee behavioural objectives 
objective test, 64 
objectivity of tests, 51-2 
open book test, 75 
oral assessment, 74, 81 



performance, 98 
performance levels, 24 
phi co-efficient, 118-121 
practical assessment, 54-57 

checklists, 56 

self assessment, 57 
profile reporting, 139-155 
project and field work assignments, 57, 80 

choosing projects, 58 

group work, 58 

role of teachers, 58 

scope, 58 
predictive validity, 128 
psychomotor domain, 25-6, 40, 54-5 
psychomotor skills, 42 
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qualitative evaluationr 8 
quality of './orkmanship/ 55 
quantitative evaluation, 7 



reliability, 125-6, 129 
co-efficient, 130 
error, 130 

of norm-referenced assessment, 13 

of criterion-referenced assessment, 
reporting marks and grades, 138 
reporting results, 133 

in TAFE, 133 
report profiles, 139-155 
reports, 133 

and admissions officers, 137 

and composite scores, 138*9 

and employers, 137 

and parents, 135 

and standards, 138 

and students, 134 

and teachers, 137 

audiences for, 134 
research, 8 



scores, 13 

selection, 128 

self assessment, 57 

short answer questions, 63 

guidelines for preparing, 64 
standards, 55, 98, 138 
statistics (of tests), 109 

for criterion-referenced tests, 114 

for norm-referenced tests, 110 

storage, 113, 115 
stem, 65 

stimulus material, 65 
structured essay, 62 

guidelines for preparing, 63 
summative assessments, 9 
summative evaluation, 8 
synthesis items, 27, 36-7 



162 



table of specifications, 96-7, 100-l^Jl 
test construction, 97 

use of time, 52-3 
test planning, 94 

achieving balance, 94 
test items, 51 
test questions, 51 
tests, 3, 51 

achievement, 3 

aptitude, 6 

criterion-referenced, 11, 19 

diagnostic, 7 

for class, 9 

norm-referenced , 11 , 19 
test specifications, 93-102 

assigning weightings, 96, 101 

comprehensiveness, 98 
test types, 52-4, 62-71, 82-5 

listening assessment, 54, 74 

matching pairs, 72 

multiple choice, 52, 64-71, 78, 82-5 
open-ended essay, 52, 60 
oral assessment, S4, 74 
practical assessment, 54 
project work, 54 
sentence completion, 52, 64 
short answer, 52, 63 
structured essay, 52, 62 
true-false, 71 
written assessment, 54 
true-false questions, 71 

guidelines for writing, 71-2 



validity, 109, 125-6 
and TAFE teacher, 125 
contentir 127 
face, 127 
predictive, 128 
via consensus moderation, 116 



written assessments, 59 
essay, 59 

matching pairs, 59 
multiple choice, 59 
6hort answer, 59 
structured essay, 59 
true-false, 59 



Year 12 examination, 129 
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STUDENT ASSESSMENT: A HANDBOOK -OR TAPE TEACHERS 

This book is for teachers and trainers woricing in vocational education. 

It aims to meet the needs of TAPE teachers as well as tiainers from industry and 
commerce who wish to improve their assessment techniques. 

This is a practical book with copious examples of what to <lo and what not to do 
when testing students. 

It should be equally of value to beginners and to thof<t who wish to sharpen 
thek existing skills in assessment. 

Readers mt h a knowledge of other texts on the subject will note that new ideas and 
new. cjuphases are presented. Practical assessment receives special attention as do 
new approaches to describing student performance. 

Peter Thomson has worked in the area of student assessment for the past 15 years 
hcih in Australia and overseas. He was formerly Head of the Tfest Development 
Division at the Australian Council of Educational Research (ACER) and is 
currently the Deputy Director of the TAPE National Centn for Research and 
Development. 
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